Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Karpenter nodes fail to join cluster due to missing default IAM policies #438

Open
1 task done
pbdco opened this issue Dec 8, 2024 · 1 comment
Open
1 task done
Labels

Comments

@pbdco
Copy link

pbdco commented Dec 8, 2024

Description

When using the Karpenter addon, the nodes provisioned by Karpenter fail to join the cluster due to missing essential IAM policies. The node role created by the module doesn't include the basic EKS worker node policies by default, which are required for:

  1. Authenticate with the EKS control plane
  2. Configure CNI networking
  3. Pull container images from ECR
  • ✋ I have searched the open/closed issues and my issue is not listed.

Expected behaviour

Nodes provisioned by Karpenter should automatically join the cluster with the necessary permissions to:

  1. Authenticate with the EKS control plane
  2. Configure CNI networking
  3. Pull container images from ECR

Actual behaviour

Nodes fail to join the cluster with permission errors. The issue can be fixed by manually adding the required policies:

karpenter = {
  node_iam_policy_arns = [
    "arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy",
    "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy",
    "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"
  ]
}

Error from Karpenter logs:

{"level":"error","time":"2024-12-08T21:42:15.000Z","logger":"controller.nodeclaim","message":"An error occurred (UnauthorizedOperation) when calling the DescribeInstances operation: You are not authorized to perform this operation. User: arn:aws:sts::559050245391:assumed-role/karpenter-spot-and-karpenter/i-050359de8285690aa is not authorized to perform: ec2:DescribeInstances"}

These policies should be included by default as they are essential for any EKS worker node to function properly.

Additional context

This is similar to how the EKS module includes these policies by default for managed node groups. The Karpenter module should follow the same pattern to ensure nodes can join the cluster out of the box.

Suggested fix: Add these three policies as defaults in the module's main.tf, while still allowing users to override or add additional policies through the node_iam_policy_arns variable.

Copy link

github-actions bot commented Jan 8, 2025

This issue has been automatically marked as stale because it has been open 30 days
with no activity. Remove stale label or comment or this issue will be closed in 10 days

@github-actions github-actions bot added the stale label Jan 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant