Best way to mount secrets as env vars?

[jmeisele]

More of a question than an issue, I am pulling secrets from vault and writing them out to /vault/secrets/secrets.env
during a vault-agent init container process. What would be the best way to load this file as environment variables in my prefect worker using the helm chart provided?

Example below

FOO=bar

[jawnsy]

Hey there, thanks so much for your question! We don't currently support customization of the entrypoint, which is what you would need to do to load secrets from a file:

prefect-helm/charts/prefect-worker/templates/deployment.yaml

Lines 58 to 62 in e94f1d4

	command:
	- /usr/bin/tini
	- -g
	- --
	- /opt/prefect/entrypoint.sh

If you can use the Vault CSI Provider instead, then you should be able to customize the volumeMounts and volumes using the extraVolumeMounts and extraVolumes Helm values, respectively. This should have similar security considerations as using the vault-agent injector, since secrets are stored in an ephemeral volume using the service account credentials of the pod.

We can consider adding support for a customizable entrypoint (and we would be happy to review a pull request adding that capability), but we would need to be clear in our documentation that it is an unsupported configuration, since we cannot test every possible configuration, and customizing this necessarily requires that users understand certain implementation details of the worker.

An alternative that would allow you to override the entrypoint today is to copy the entrypoint script contents to a ConfigMap and customize it as you wish, then mount it on top of the existing entrypoint script using extraVolumes/extraVolumeMounts. We're happy to provide some assistance for you to do this, but please keep in mind that you will take on some additional maintenance to make sure this keeps working (in practice, we have rarely modified this script, but that is not guaranteed indefinitely.)

[jmeisele]

@jawnsy Appreciate the quick response, I was hoping to avoid tampering with entrypoint.sh at all costs. Appreciate the 2nd set of eyes on this

[jmeisele]

@jawnsy and anyone else looking for an easy fix. These are the annotations added to the helm chart I went with. This pokes vault and injects the secret as a file in /vault/secrets/secret.env. The first task in my flow uses the dotenv library to load them 😄

      annotations:
        vault.hashicorp.com/agent-inject: 'true'
        vault.hashicorp.com/role: 'some-role'
        vault.hashicorp.com/agent-inject-secret-secrets.env: 'path/to/secret'
        # Environment variable export template
        vault.hashicorp.com/agent-inject-template-secrets.env: |
          {{- with secret "path/to/secret" -}}
            FOO="{{ .Data.data.bar }}"
          {{- end }}

import os
from typing import Union
from dotenv import load_dotenv

from prefect import flow, task, get_run_logger

@task(name="Load env variables from secrets")
def load_env_var() -> None:
    load_dotenv("/vault/secrets/secrets.env")

@task(name="Grab env variable")
def fetch_env_var() -> Union[str, None]:
    return os.getenv("FOO")

@flow(name="Echo environment variable")
def env_var_flow() -> None:
    logger = get_run_logger()
    load_env_var()
    result = fetch_env_var()
    logger.info(f"FOO env variable set as: {result}")