IAM roles for service accounts vs kube2iam

[jtblin]

With IAM roles for service accounts, my understanding was that kube2iam was not required anymore and that IAM roles for service accounts should be preferred for kubernetes on AWS.

It doesn't look like it's 100% the case so I am trying to understand why kube2iam is still needed. Are IAM roles for service accounts only for EKS? Are there reasons to prefer kube2iam even on EKS? I would love to hear the thoughst of the community to help guiding the future direction for kube2iam e.g. deprecation or not.

[davidegiunchi]

I'm using EKS on AWS and ~ 1 and half years ago i've successfully migrated from kube2iam to IRSA.
If you are using EKS, IRSA is surely a drop-in-replacement for kube2iam and i don't think that there's a reason to prefer kube2iam.
I think that it works even on manually provisioned K8S clusters on EC2:
https://dev.to/olemarkus/zero-configuration-irsa-on-kops-1po1
https://kops.sigs.k8s.io/cluster_spec/#service-account-issuer-discovery-and-aws-iam-roles-for-service-accounts-irsa

[mindgrep]

Firstly, thank you very much for the work on kube2iam, we have been using it in EKS for a while now in production. We started looking at IAM roles for service accounts recently because of the lack of support for IMDSv2 (now fixed, thanks to you and to the author of that PR for that), and also because until recently, this project appeared to have stopped being actively supported (understandably so).

But IAM roles for service accounts provided by AWS has been anything but smooth for us in adoption.
They basically provide 2 ways to use them, for cross-account IAM role access (which is our main use case), as documented here:

1. Creating an identity provider from another account's cluster or
2. Using chained AssumeRole operations

The first option is basically discouraged (blocked) by our corporate account governance due to security concerns, so we are left with option 2. With option 2, it seems like AWS has provided a half-baked solution - they provide an admission controller that injects env vars needed to make role assumption work for a container running in EKS. However, this does not suffice for cross account role assumption, which uses IAM role chaining.

For role chaining to work, the running container has to know both the local role and the "chained" remote account role arns. So for us as a platform team, we would need to have every single tenant update their deployments to inject these values into their deployments.. OR for us to create some tooling (like a controller) to inject these automatically.

So this makes staying with kube2iam a much more preferred option :) as long as it supports our current infrastructure.
Since IMDSv2 is now supported, we can consider staying with kube2iam. However, we are migrating to ipv6 networking for pods.. and currently kube2iam does not support ipv6 "out of the box" (it uses iptables which is ipv4 only).

We would not mind PRing support for ipv6 to the project, if that is something you would consider.
The underlying go iptables library does support ipv4 as well as ipv6.

[mischmi2]

We still use kube2iam because IAM roles for service accounts require exposure of unauthenticated public json oauth provider data in an S3 bucket or a server, and our organization policies prohibit such things, meaning the entire integration AWS has set out isn't accessible to us. Very similar to @mindgrep 's comment about corporate governance.

[benjimin]

There's a post here outlining several alternatives that presently exist (including IRSA and, more recently, Pod Identities).

[stepherickson71]

What about customers running clusters in AWS on straight EC2 instances using kubeadm as setup method, for example to avoid a publicly exposed kube-api endpoint?
update: IRSA is supported on non-EKS clusters it looks like

[benjimin]

Yep, IRSA just means using the OIDC identity provider (and token volume projection) that is already built into k8s, so that pod processes can prove their service account associations to any other relying system (in this case specifically to STS, which is the AWS service for converting external/federated authorisation tokens into temporary IAM credentials; anything built from the AWS SDK will automatically handle all this without any container configuration). This is the same mechanism used by k8s internally for API authentication; it’s not EKS specific, which may (cynically) be one reason why AWS has additional introduced EKS Pod Identities (a proprietary alternative mechanism).