use an aws role arn (irsa) while still loading the webIdenityToken from a file

[clayrisser]

When configuring an application with IAM roles for service accounts (IRSA) in a Kubernetes environment, you should not need to manually supply the Web Identity Token in your configuration. This token is dynamically generated and injected into your pod by the EKS environment.

The token should be loaded from a file.

https://github.com/kanisterio/kanister/blob/master/pkg/aws/aws.go#L132

I see that there is an option to load the webIdenityToken from a file, but that only works when configured using ENVs.

https://github.com/kanisterio/kanister/blob/master/pkg/aws/aws.go#L105

It should be possible to use an aws role arn (irsa) while still loading the webIdenityToken from a file.

My proposed solution is that instead of returning nil, nil when a webIdenityToken is not found, simply check to see if it's located at /var/run/secrets/eks.amazonaws.com/serviceaccount/token first.

https://github.com/kanisterio/kanister/blob/master/pkg/aws/aws.go#L126

[github-actions]

Thanks for opening this issue 👍. The team will review it shortly.

If this is a bug report, make sure to include clear instructions how on to reproduce the problem with minimal reproducible examples, where possible. If this is a security report, please review our security policy as outlined in SECURITY.md.

If you haven't already, please take a moment to review our project's Code of Conduct document.

[clayrisser]

In addition I think it would need to add the eks.amazonaws.com/role-arn annotation to the job.

[viveksinghggits]

cc @hairyhum

[leuyentran]

hi @clayrisser , I think I need your help to understand better the scenario being described here 😄

My question is whether or not the description below is the same set up that you are using? If not maybe we are missing a third use case!

TL,DR: Kanister is already checking for /var/run/secrets/eks.amazonaws.com/serviceaccount/token! It is done via env var AWS_WEB_IDENTITY_TOKEN_FILE if IRSA and Amazon EKS Pod Identity Webhook are set up

Long explantion:

When configuring an application with IAM roles for service accounts (IRSA) in a Kubernetes environment, you should not need to manually supply the Web Identity Token in your configuration. This token is dynamically generated and injected into your pod by the EKS environment.

The token should be loaded from a file.

https://github.com/kanisterio/kanister/blob/master/pkg/aws/aws.go#L132

You are absolutely right that when IRSA is used, there shouldn't be a need to manually supply for Web Identity Token because the token should be requested and stored by the kubelet on behalf of the Pod (src).

Additionally, if running on EKS clusters, the Amazon EKS Pod Identity Webhook watches for Pods that use a K8s Service Account annotated with eks.amazonaws.com/role-arn: arn:aws:iam::111122223333:role/my-role and then injects several ENV_VAR into these Pods, including :
-AWS_WEB_IDENTITY_TOKEN_FILE that points to where the token file should be store, i.e. /var/run/secrets/eks.amazonaws.com/serviceaccount/token
-AWS_ROLE_ARN which is the AWS role that should be assumed. This is the same role that the K8s Service Account is annotated viaeks.amazonaws.com/role-arn

The scenario with Amazon EKS Pod Identity Webhook above is what the function fetchWebIdentityTokenFromFile accounts for. (Albeit that is actually not the best function name 😅 )

Which means the statement below is true, however the environment variables configuration part should be done by Amazon EKS Pod Identity Webhook on behalf of the Pod, and there should not be any additional work from user beside setting up (step 1, 2, and 3 here)

I see that there is an option to load the webIdenityToken from a file, but that only works when configured using ENVs.

So going back to the bugfix suggestion:

It should be possible to use an aws role arn (irsa) while still loading the webIdenityToken from a file.

This is entirely the intention of the caller function authenticateAWSCredentials which calls fetchWebIdentityTokenFromConfig and fetchWebIdentityTokenFromFile successively.

My proposed solution is that instead of returning nil, nil when a webIdenityToken is not found, simply check to see if it's located at /var/run/secrets/eks.amazonaws.com/serviceaccount/token first.

And checking for a token located at /var/run/secrets/eks.amazonaws.com/serviceaccount/token should be achieved with fetchWebIdentityTokenFromFile if the cluster and Pod are set up to use ISRA and Amazon EKS Pod Identity Webhook

[clayrisser]

Ok, so it works with irsa? Do you have any examples?

[hairyhum]

@clayrisser there are no examples to demonstrate this specific behaviour, but what you can do is to create a KubeTask function in your blueprint and specify podOverride there to contain serviceAccountName that you need:

- func: KubeTask
  name: examplePhase
  args:
    namespace: "{{ .Deployment.Namespace }}"
    image: busybox
    podOverride:
      serviceAccountName: my-serviceaccount
    command:
      - sh
      - -c
      - |
        aws sts get-caller-identity

If the pod is created in EKS and credentials injection is properly set up, you should get the role for the service account you're requesting.