Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Update EKS module to support AL2023 bootstrapping #689

Merged
merged 3 commits into from
Oct 30, 2024

Conversation

alanty
Copy link
Contributor

@alanty alanty commented Oct 30, 2024

What does this PR do?

This PR updates the EKS terraform module to bring in support for NodeADM bootstrapping.
it also updates the IAM configuration for Karpenter to use access_entries instead of aws_auth_roles.

Motivation

#688 added MNG using AL2023 AMIs which require newer Bootstrapping:


      cloudinit_pre_nodeadm = [
        {
          content_type = "application/node.eks.aws"
          content      = <<-EOT
            ---
            apiVersion: node.eks.aws/v1alpha1
            kind: NodeConfig
            spec:
              instance:
                localStorage:
                  strategy: RAID0
          EOT
        }
      ]

This cloudinit_pre_nodeadm property isn't supported in version 19.x of the EKS terraform module, and the aws_auth entries we use are removed in 20.x+

More

  • Yes, I have tested the PR using my local account setup (Provide any test evidence report under Additional Notes)
  • Mandatory for new blueprints. Yes, I have added a example to support my blueprint PR
  • Mandatory for new blueprints. Yes, I have updated the website/docs or website/blog section for this feature
  • Yes, I ran pre-commit run -a with this PR. Link for installing pre-commit locally

For Moderators

  • E2E Test successfully complete before merge?

Additional Notes

I was able to check the ephemeral storage thats being returned from the spark_benchmark_ssd

Capacity:
  cpu:                48
  ephemeral-storage:  1756689344Ki
  hugepages-1Gi:      0
  hugepages-2Mi:      0
  memory:             96758624Ki
  pods:               234

1756689344Ki == 1675.3Gi which matches the NVMe space

checking the volumes on the node and it looks like the RAID was setup correctly

[root@ip-100-64-176-24 bin]# df -h
Filesystem        Size  Used Avail Use% Mounted on
devtmpfs          4.0M     0  4.0M   0% /dev
tmpfs              47G     0   47G   0% /dev/shm
tmpfs              19G  1.9M   19G   1% /run
/dev/nvme0n1p1    100G  2.8G   98G   3% /
tmpfs              47G     0   47G   0% /tmp
/dev/nvme0n1p128   10M  1.3M  8.7M  13% /boot/efi
/dev/md127        1.7T   13G  1.7T   1% /mnt/k8s-disks/0

@alanty alanty requested a review from vara-bonthu October 30, 2024 18:32
@alanty alanty changed the title [bug] update EKS module to support AL2023 bootstrapping [fix] update EKS module to support AL2023 bootstrapping Oct 30, 2024
@alanty alanty changed the title [fix] update EKS module to support AL2023 bootstrapping [fix] Update EKS module to support AL2023 bootstrapping Oct 30, 2024
@alanty alanty changed the title [fix] Update EKS module to support AL2023 bootstrapping fix: Update EKS module to support AL2023 bootstrapping Oct 30, 2024
Copy link
Collaborator

@vara-bonthu vara-bonthu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@vara-bonthu vara-bonthu merged commit c624001 into awslabs:main Oct 30, 2024
37 of 41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants