Skip to content

Commit

Permalink
Add debugging steps for DiscoverEKS User Task issues (#50377)
Browse files Browse the repository at this point in the history
This PR adds description to all known discover EKS issues.

It also adds URLs that the user can follow to fix or further debug the
issue.
The URLs are per-cluster, and they usually link to Amazon EKS Cluster
page or one of its sub pages.
  • Loading branch information
marcoandredinis authored Jan 9, 2025
1 parent c6e442f commit 588d024
Show file tree
Hide file tree
Showing 10 changed files with 385 additions and 7 deletions.
19 changes: 15 additions & 4 deletions lib/usertasks/descriptions.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,14 +26,25 @@ import (
//go:embed descriptions/*.md
var descriptionsFS embed.FS

// DescriptionForDiscoverEC2Issue returns the description of the issue and fixing steps.
// The returned string contains a markdown document.
// If issue type is not recognized or doesn't have a specific description, them an empty string is returned.
func DescriptionForDiscoverEC2Issue(issueType string) string {
func loadIssueDescription(issueType string) string {
filename := fmt.Sprintf("descriptions/%s.md", issueType)
bs, err := descriptionsFS.ReadFile(filename)
if err != nil {
return ""
}
return string(bs)
}

// DescriptionForDiscoverEC2Issue returns the description of the issue and fixing steps.
// The returned string contains a markdown document.
// If issue type is not recognized or doesn't have a specific description, them an empty string is returned.
func DescriptionForDiscoverEC2Issue(issueType string) string {
return loadIssueDescription(issueType)
}

// DescriptionForDiscoverEKSIssue returns the description of the issue and fixing steps.
// The returned string contains a markdown document.
// If issue type is not recognized or doesn't have a specific description, them an empty string is returned.
func DescriptionForDiscoverEKSIssue(issueType string) string {
return loadIssueDescription(issueType)
}
8 changes: 8 additions & 0 deletions lib/usertasks/descriptions/eks-agent-not-connecting.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
The process of automatically enrolling EKS Clusters into Teleport, starts by installing the [`teleport-kube-agent`](https://goteleport.com/docs/reference/helm-reference/teleport-kube-agent/) to the cluster.

If the installation is successful, the EKS Cluster will appear in your Resources list.

However, the following EKS Clusters did not automatically enrolled.
This usually happens when the installation is taking too long or there was an error preventing the HELM chart installation.

Open the Teleport Agent to get more information.
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Teleport uses the Amazon EKS API to install the Teleport Kubernetes Agent.

Please enable API (or API and Config Map) authentication mode in the following EKS Clusters so that they can be automatically enrolled.
5 changes: 5 additions & 0 deletions lib/usertasks/descriptions/eks-cluster-unreachable.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
The EKS Cluster must be accessible from the Teleport Auth Service in order for Teleport to deploy the Teleport Kubernetes Agent.

The following EKS Clusters couldn't be accessed.

Ensure their network endpoint access configuration allows access from Teleport.
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
The EKS Cluster must be publicly accessible in order for Teleport to deploy the Teleport Kubernetes Agent.

You can enable the public endpoint by accessing the Manage Endpoint Access.
3 changes: 3 additions & 0 deletions lib/usertasks/descriptions/eks-status-not-active.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Only EKS Clusters whose status is active can be automatically enrolled into teleport.

The following are not active.
3 changes: 3 additions & 0 deletions lib/usertasks/descriptions_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,4 +30,7 @@ func TestAllDescriptions(t *testing.T) {
for _, issueType := range usertasksapi.DiscoverEC2IssueTypes {
require.NotEmpty(t, DescriptionForDiscoverEC2Issue(issueType), "issue type %q is missing descriptions/%s.md file", issueType, issueType)
}
for _, issueType := range usertasksapi.DiscoverEKSIssueTypes {
require.NotEmpty(t, DescriptionForDiscoverEKSIssue(issueType), "issue type %q is missing descriptions/%s.md file", issueType, issueType)
}
}
174 changes: 174 additions & 0 deletions lib/usertasks/urls.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,174 @@
/*
* Teleport
* Copyright (C) 2025 Gravitational, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/

package usertasks

import (
"net/url"
"path"

usertasksv1 "github.com/gravitational/teleport/api/gen/proto/go/teleport/usertasks/v1"
usertasksapi "github.com/gravitational/teleport/api/types/usertasks"
)

// UserTaskDiscoverEKSWithURLs contains the clusters that failed to auto-enroll into the cluster.
type UserTaskDiscoverEKSWithURLs struct {
*usertasksv1.DiscoverEKS
// Clusters maps a cluster name to the result of enrolling that cluster into teleport.
Clusters map[string]*DiscoverEKSClusterWithURLs `json:"clusters,omitempty"`
}

// DiscoverEKSClusterWithURLs contains the result of enrolling an AWS EKS Cluster.
type DiscoverEKSClusterWithURLs struct {
*usertasksv1.DiscoverEKSCluster

// ResourceURL is the Amazon Web Console URL to access this EKS Cluster.
// Always present.
// Format: https://console.aws.amazon.com/eks/home?region=<region>#/clusters/<cluster-name>
ResourceURL string `json:"resourceUrl,omitempty"`

// OpenTeleportAgentURL is the URL to open the Teleport Agent StatefulSet in Amazon EKS Web Console.
// Present when issue is of type eks-agent-not-connecting.
// Format: https://console.aws.amazon.com/eks/home?region=<region>#/clusters/<cluster-name>/statefulsets/teleport-kube-agent?namespace=teleport-agent
OpenTeleportAgentURL string `json:"openTeleportAgentUrl,omitempty"`

// ManageAccessURL is the URL to open the EKS in Amazon Web Console, in the Manage Access page.
// Present when issue is of type eks-authentication-mode-unsupported.
// Format: https://console.aws.amazon.com/eks/home?region=<region>#/clusters/<cluster-name>/manage-access
ManageAccessURL string `json:"manageAccessUrl,omitempty"`

// ManageEndpointAccessURL is the URL to open the EKS in Amazon Web Console, in the Manage Endpoint Access page.
// Present when issue is of type eks-cluster-unreachable and eks-missing-endpoint-public-access.
// Format: https://console.aws.amazon.com/eks/home?region=<region>#/clusters/<cluster-name>/manage-endpoint-access
ManageEndpointAccessURL string `json:"manageEndpointAccessUrl,omitempty"`

// ManageClusterURL is the URL to open the EKS Cluster in Amazon Web Console.
// Present when issue is of type eks-status-not-active.
// Format: https://console.aws.amazon.com/eks/home?region=<region>#/clusters/<cluster-name>
ManageClusterURL string `json:"manageClusterUrl,omitempty"`
}

func withEKSClusterIssueURL(metadata *usertasksv1.UserTask, cluster *usertasksv1.DiscoverEKSCluster) *DiscoverEKSClusterWithURLs {
ret := &DiscoverEKSClusterWithURLs{
DiscoverEKSCluster: cluster,
}
clusterBaseURL := url.URL{
Scheme: "https",
Host: "console.aws.amazon.com",
Path: path.Join("eks", "home"),
Fragment: "/clusters/" + cluster.GetName(),
RawQuery: url.Values{
"region": []string{metadata.Spec.DiscoverEks.GetRegion()},
}.Encode(),
}

ret.ResourceURL = clusterBaseURL.String()

switch metadata.Spec.IssueType {
case usertasksapi.AutoDiscoverEKSIssueAgentNotConnecting:
clusterBaseURL.Fragment = clusterBaseURL.Fragment + "/statefulsets/teleport-kube-agent?namespace=teleport-agent"
ret.OpenTeleportAgentURL = clusterBaseURL.String()

case usertasksapi.AutoDiscoverEKSIssueAuthenticationModeUnsupported:
clusterBaseURL.Fragment = clusterBaseURL.Fragment + "/manage-access"
ret.ManageAccessURL = clusterBaseURL.String()

case usertasksapi.AutoDiscoverEKSIssueClusterUnreachable, usertasksapi.AutoDiscoverEKSIssueMissingEndpoingPublicAccess:
clusterBaseURL.Fragment = clusterBaseURL.Fragment + "/manage-endpoint-access"
ret.ManageEndpointAccessURL = clusterBaseURL.String()

case usertasksapi.AutoDiscoverEKSIssueStatusNotActive:
ret.ManageClusterURL = clusterBaseURL.String()
}

return ret
}

// EKSClustersWithURLs takes a UserTask and enriches the cluster list with URLs.
// Currently, the following URLs will be added:
// - ResourceURL: a link to open the instance in Amazon Web Console.
// The following URLs might be added depending on the issue type:
// - OpenTeleportAgentURL: links directly to the statefulset created during the helm installation
// - ManageAccessURL: links to the Manage Access screen in the Amazon EKS Web Console, for the current EKS Cluster.
// - ManageEndpointAccessURL: links to the Manage Endpoint Access screen in the Amazon EKS Web Console, for the current EKS Cluster.
// - ManageClusterURL: links to the EKS Cluster.
func EKSClustersWithURLs(ut *usertasksv1.UserTask) *UserTaskDiscoverEKSWithURLs {
clusters := ut.Spec.GetDiscoverEks().GetClusters()
clustersWithURLs := make(map[string]*DiscoverEKSClusterWithURLs, len(clusters))

for clusterName, cluster := range clusters {
clustersWithURLs[clusterName] = withEKSClusterIssueURL(ut, cluster)
}

return &UserTaskDiscoverEKSWithURLs{
DiscoverEKS: ut.Spec.GetDiscoverEks(),
Clusters: clustersWithURLs,
}
}

// UserTaskDiscoverEC2WithURLs contains the instances that failed to auto-enroll into the cluster.
type UserTaskDiscoverEC2WithURLs struct {
*usertasksv1.DiscoverEC2
// Instances maps the instance ID name to the result of enrolling that instance into teleport.
Instances map[string]*DiscoverEC2InstanceWithURLs `json:"clusters,omitempty"`
}

// DiscoverEC2InstanceWithURLs contains the result of enrolling an AWS EC2 Instance.
type DiscoverEC2InstanceWithURLs struct {
*usertasksv1.DiscoverEC2Instance

// ResourceURL is the Amazon Web Console URL to access this EC2 Instance.
// Always present.
// Format: https://console.aws.amazon.com/ec2/home?region=<region>#InstanceDetails:instanceId=<instance-id>
ResourceURL string `json:"resourceUrl,omitempty"`
}

func withEC2InstanceIssueURL(metadata *usertasksv1.UserTask, instance *usertasksv1.DiscoverEC2Instance) *DiscoverEC2InstanceWithURLs {
ret := &DiscoverEC2InstanceWithURLs{
DiscoverEC2Instance: instance,
}
instanceBaseURL := url.URL{
Scheme: "https",
Host: "console.aws.amazon.com",
Path: path.Join("ec2", "home"),
Fragment: "InstanceDetails:instanceId=" + instance.GetInstanceId(),
RawQuery: url.Values{
"region": []string{metadata.Spec.DiscoverEc2.GetRegion()},
}.Encode(),
}
ret.ResourceURL = instanceBaseURL.String()

return ret
}

// EC2InstancesWithURLs takes a UserTask and enriches the instance list with URLs.
// Currently, the following URLs will be added:
// - ResourceURL: a link to open the instance in Amazon Web Console.
func EC2InstancesWithURLs(ut *usertasksv1.UserTask) *UserTaskDiscoverEC2WithURLs {
instances := ut.Spec.GetDiscoverEc2().GetInstances()
instancesWithURLs := make(map[string]*DiscoverEC2InstanceWithURLs, len(instances))

for instanceID, instance := range instances {
instancesWithURLs[instanceID] = withEC2InstanceIssueURL(ut, instance)
}

return &UserTaskDiscoverEC2WithURLs{
DiscoverEC2: ut.Spec.GetDiscoverEc2(),
Instances: instancesWithURLs,
}
}
151 changes: 151 additions & 0 deletions lib/usertasks/urls_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
/*
* Teleport
* Copyright (C) 2025 Gravitational, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/

package usertasks

import (
"testing"

"github.com/stretchr/testify/require"

usertasksv1 "github.com/gravitational/teleport/api/gen/proto/go/teleport/usertasks/v1"
usertasksapi "github.com/gravitational/teleport/api/types/usertasks"
)

func TestEKSURLs(t *testing.T) {
clusterName := "my-cluster"
dummyCluster := &usertasksv1.DiscoverEKSCluster{Name: clusterName}
baseClusterData := &usertasksv1.DiscoverEKS{
Region: "us-east-1",
Clusters: map[string]*usertasksv1.DiscoverEKSCluster{
clusterName: dummyCluster,
},
}

for _, tt := range []struct {
name string
issueType string
expectedEKSClusterWithURL *DiscoverEKSClusterWithURLs
expected *UserTaskDiscoverEKSWithURLs
}{
{
name: "url for eks agent not connecting",
issueType: usertasksapi.AutoDiscoverEKSIssueAgentNotConnecting,
expectedEKSClusterWithURL: &DiscoverEKSClusterWithURLs{
ResourceURL: "https://console.aws.amazon.com/eks/home?region=us-east-1#/clusters/my-cluster",
OpenTeleportAgentURL: "https://console.aws.amazon.com/eks/home?region=us-east-1#/clusters/my-cluster/statefulsets/teleport-kube-agent?namespace=teleport-agent",
},
},
{
name: "url for eks authentication mode unsupported",
issueType: usertasksapi.AutoDiscoverEKSIssueAuthenticationModeUnsupported,
expectedEKSClusterWithURL: &DiscoverEKSClusterWithURLs{
ResourceURL: "https://console.aws.amazon.com/eks/home?region=us-east-1#/clusters/my-cluster",
ManageAccessURL: "https://console.aws.amazon.com/eks/home?region=us-east-1#/clusters/my-cluster/manage-access",
},
},
{
name: "url for eks cluster unreachable",
issueType: usertasksapi.AutoDiscoverEKSIssueClusterUnreachable,
expectedEKSClusterWithURL: &DiscoverEKSClusterWithURLs{
ResourceURL: "https://console.aws.amazon.com/eks/home?region=us-east-1#/clusters/my-cluster",
ManageEndpointAccessURL: "https://console.aws.amazon.com/eks/home?region=us-east-1#/clusters/my-cluster/manage-endpoint-access",
},
},
{
name: "url for eks missing endpoint public access",
issueType: usertasksapi.AutoDiscoverEKSIssueMissingEndpoingPublicAccess,
expectedEKSClusterWithURL: &DiscoverEKSClusterWithURLs{
ResourceURL: "https://console.aws.amazon.com/eks/home?region=us-east-1#/clusters/my-cluster",
ManageEndpointAccessURL: "https://console.aws.amazon.com/eks/home?region=us-east-1#/clusters/my-cluster/manage-endpoint-access",
},
},
{
name: "url for eks cluster status not active",
issueType: usertasksapi.AutoDiscoverEKSIssueStatusNotActive,
expectedEKSClusterWithURL: &DiscoverEKSClusterWithURLs{
ResourceURL: "https://console.aws.amazon.com/eks/home?region=us-east-1#/clusters/my-cluster",
ManageClusterURL: "https://console.aws.amazon.com/eks/home?region=us-east-1#/clusters/my-cluster",
},
},
} {
t.Run(tt.name, func(t *testing.T) {
clusterWithURL := tt.expectedEKSClusterWithURL
clusterWithURL.DiscoverEKSCluster = dummyCluster
expected := &UserTaskDiscoverEKSWithURLs{
DiscoverEKS: baseClusterData,
Clusters: map[string]*DiscoverEKSClusterWithURLs{
clusterName: clusterWithURL,
},
}

got := EKSClustersWithURLs(&usertasksv1.UserTask{
Spec: &usertasksv1.UserTaskSpec{
IssueType: tt.issueType,
DiscoverEks: baseClusterData,
},
})
require.Equal(t, expected, got)
})
}
}

func TestEC2URLs(t *testing.T) {
instanceID := "i-12345678"
dummyInstance := &usertasksv1.DiscoverEC2Instance{InstanceId: instanceID}
baseInstancesData := &usertasksv1.DiscoverEC2{
Region: "us-east-1",
Instances: map[string]*usertasksv1.DiscoverEC2Instance{
instanceID: dummyInstance,
},
}

for _, tt := range []struct {
name string
issueType string
expectedEC2InstanceWithURL *DiscoverEC2InstanceWithURLs
expected *UserTaskDiscoverEC2WithURLs
}{
{
name: "url for ec2 resource",
issueType: usertasksapi.AutoDiscoverEC2IssueSSMScriptFailure,
expectedEC2InstanceWithURL: &DiscoverEC2InstanceWithURLs{
ResourceURL: "https://console.aws.amazon.com/ec2/home?region=us-east-1#InstanceDetails:instanceId=i-12345678",
},
},
} {
t.Run(tt.name, func(t *testing.T) {
instanceWithURL := tt.expectedEC2InstanceWithURL
instanceWithURL.DiscoverEC2Instance = dummyInstance
expected := &UserTaskDiscoverEC2WithURLs{
DiscoverEC2: baseInstancesData,
Instances: map[string]*DiscoverEC2InstanceWithURLs{
instanceID: instanceWithURL,
},
}

got := EC2InstancesWithURLs(&usertasksv1.UserTask{
Spec: &usertasksv1.UserTaskSpec{
IssueType: tt.issueType,
DiscoverEc2: baseInstancesData,
},
})
require.Equal(t, expected, got)
})
}
}
Loading

0 comments on commit 588d024

Please sign in to comment.