Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix broken outputs from GSM actions #224

Open
wants to merge 38 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
91a1fdd
fix: bump to 1.49.0 to avoid symlink issue
DerTiedemann Nov 26, 2024
9409185
fix: update to new action name
DerTiedemann Nov 26, 2024
4e67ea6
fix: use fixed branch
DerTiedemann Nov 26, 2024
ba11055
fix: learn how to write
DerTiedemann Nov 26, 2024
3bbd7e8
fix: make docker work
DerTiedemann Nov 26, 2024
33110a1
fix: workspace work differently in GH actions
DerTiedemann Nov 26, 2024
92024a1
test: try without workspace
DerTiedemann Nov 26, 2024
e8ae8eb
debug: weird gh actions env
DerTiedemann Nov 26, 2024
32bca0f
debug: only pwd cuz i dont know how cmds work (i do but i forgor)
DerTiedemann Nov 26, 2024
9491d02
debug: see i know how it works
DerTiedemann Nov 26, 2024
bcc85b2
debug: set workdir again
DerTiedemann Nov 26, 2024
4f904c8
debug: try sth different
DerTiedemann Nov 26, 2024
f22e071
debug: try other random stuff
DerTiedemann Nov 26, 2024
1a8dd55
fix: cd into workdir because github madness
DerTiedemann Nov 26, 2024
0d3bdc2
fix: stringify secret outputs
DerTiedemann Nov 26, 2024
0f83dc1
fix: dont use secret subobject
DerTiedemann Nov 26, 2024
337eaef
fix: try again with JSON
DerTiedemann Nov 26, 2024
66f4c24
fix: secret plumbing
DerTiedemann Nov 27, 2024
68f61ee
fix: now there is javascript
DerTiedemann Nov 27, 2024
fed5272
fix: tests
DerTiedemann Nov 27, 2024
8f92b1b
debug: try if bug is here
DerTiedemann Nov 27, 2024
05cb777
debug: add final newline
DerTiedemann Nov 27, 2024
c8305c2
debug: uncomment secret names + add debug output
DerTiedemann Nov 27, 2024
5929cd5
fix: specify bash shell
DerTiedemann Nov 27, 2024
fec5380
fix: add new line to secret-names output
DerTiedemann Nov 27, 2024
94084cb
debug: test GH string interpolation
DerTiedemann Nov 27, 2024
3c06aa3
debug: just see if basic stuff works
DerTiedemann Nov 27, 2024
4713020
debug: i thought semicolons are optional in js
DerTiedemann Nov 27, 2024
0da35dd
debug: maybe comment are also not allowed
DerTiedemann Nov 27, 2024
75d344c
debug: now try again with string
DerTiedemann Nov 27, 2024
3a802c7
fix: i forgor a bracket
DerTiedemann Nov 27, 2024
4620740
debug: what is evenhappening
DerTiedemann Nov 27, 2024
c124c48
debug: be more direct
DerTiedemann Nov 27, 2024
af63a83
fix: export all secrets
DerTiedemann Nov 27, 2024
3dd7252
debug: add some logging so see sth happening
DerTiedemann Nov 27, 2024
923224c
debug: remove defined outputs
DerTiedemann Nov 27, 2024
b6c9ff3
debug: does basics work?
DerTiedemann Nov 27, 2024
afa2354
fix: sadly i have to use this workaround
DerTiedemann Nov 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions actions/gcp-gsm-load-secrets/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ inputs:
required: false
default: true
outputs:
secrets:
description: "Secrets loaded from Secret Manager"
value: ${{ steps.secrets.outputs.secrets }}
secret-json-string:
description: "JSON string with all secrets"
value: ${{ toJSON(steps.secrets.outputs) }}
runs:
using: "composite"
steps:
Expand All @@ -31,7 +31,7 @@ runs:
workload_identity_provider: ${{ inputs.workload-identity-provider }}
service_account: ${{ inputs.gke-service-account }}
- id: "parse_secrets"
uses: "bakdata/ci-templates/actions/parse-secrets[email protected]"
uses: "bakdata/ci-templates/actions/gcp-gsm-parse-secrets@tiedemann/gsm-object-outputs-fix"
with:
project_name: ${{ inputs.gke-project-name }}
secrets_list: ${{ inputs.secrets-to-inject }}
Expand Down
21 changes: 5 additions & 16 deletions actions/gcp-gsm-parse-secrets/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,26 +1,15 @@
FROM python:3-slim AS builder
WORKDIR /action/workspace/

RUN pip install poetry==1.8.2

ENV POETRY_NO_INTERACTION=1 \
POETRY_VIRTUALENVS_IN_PROJECT=1 \
POETRY_VIRTUALENVS_CREATE=1 \
POETRY_CACHE_DIR=/tmp/poetry_cache

WORKDIR /app
COPY pyproject.toml poetry.lock ./
COPY main.py ./
COPY pyproject.toml poetry.lock /action/workspace/
COPY main.py /action/workspace/
RUN poetry install --no-root && rm -rf $POETRY_CACHE_DIR

# A distroless container image with Python and some basics like SSL certificates
# https://github.com/GoogleContainerTools/dis/i/itroless
FROM gcr.io/distroless/python3-debian12

ENV VIRTUAL_ENV=/app/.venv \
PATH="/app/.venv/bin:$PATH"

COPY --from=builder /app /app
COPY --from=builder ${VIRTUAL_ENV} ${VIRTUAL_ENV}

WORKDIR /app
ENV PYTHONPATH /app
CMD ["/app/main.py"]
CMD ["bash", "-c", "cd /action/workspace && poetry run python main.py"]
18 changes: 14 additions & 4 deletions actions/gcp-gsm-parse-secrets/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ def parse_secret(secret, project_name, delim=DEFAULT_DELIMITER):
out = f"{secret_name}:{project_name}/{components[0]}"
if len(components) == 2 and len(components[1]) != 0:
out += f"/{components[1]}"
return out
return out, secret_name


def main(
Expand All @@ -55,11 +55,21 @@ def main(
# Deduplicate the input secrets
input_secrets = set(input_secrets.splitlines())

output = ""
output = []
parsed_secret_names = []
for secret in input_secrets:
output += parse_secret(secret, gcp_project, github_output_delimter) + "\n"
parsed_secret, parsed_secret_name = parse_secret(
secret, gcp_project, github_output_delimter
)
output.append(parsed_secret)
parsed_secret_names.append(parsed_secret_name)

set_github_action_output("secrets-list", output, github_output_delimter)
set_github_action_output(
"secrets-list", "\n".join(output) + "\n", github_output_delimter
)
set_github_action_output(
"secret-names", ",".join(parsed_secret_names) + "\n", github_output_delimter
)


if __name__ == "__main__":
Expand Down
35 changes: 27 additions & 8 deletions actions/gcp-gsm-parse-secrets/tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,36 @@

from main import parse_secret


class TestParseSecret(unittest.TestCase):
def test_parse_secret(self):
self.assertEqual(parse_secret("secret_name", "project_name"), "SECRET_NAME:project_name/secret_name")
self.assertEqual(parse_secret("secret_name/version", "project_name"), "SECRET_NAME:project_name/secret_name/version")
self.assertEqual(parse_secret("123-456", "project_name"), "123_456:project_name/123-456")
self.assertEqual(parse_secret("123___123___123", "project_name"), "123_123_123:project_name/123___123___123")
self.assertEqual(parse_secret("i-like_trains__why_this?", "project_name"), "I_LIKE_TRAINS_WHY_THIS:project_name/i-like_trains__why_this?")
self.assertEqual(
parse_secret("secret_name", "project_name")[0],
"SECRET_NAME:project_name/secret_name",
)
self.assertEqual(
parse_secret("secret_name/version", "project_name")[0],
"SECRET_NAME:project_name/secret_name/version",
)
self.assertEqual(
parse_secret("123-456", "project_name")[0], "123_456:project_name/123-456"
)
self.assertEqual(
parse_secret("123___123___123", "project_name")[0],
"123_123_123:project_name/123___123___123",
)
self.assertEqual(
parse_secret("i-like_trains__why_this?", "project_name")[0],
"I_LIKE_TRAINS_WHY_THIS:project_name/i-like_trains__why_this?",
)

def test_parse_secret_special(self):
# FIXME: this test is failing and i dont know why
self.assertEqual(parse_secret("123&&123()123__123*__*_123", "project_name"), "123_123_123_123:project_name/123&&123()123__123*__*_123")
self.assertEqual(
parse_secret("123&&123()123__123*__*_123", "project_name")[0],
"123_123_123_123:project_name/123&&123()123__123*__*_123",
)


if __name__ == '__main__':
unittest.main()
if __name__ == "__main__":
unittest.main()
6 changes: 3 additions & 3 deletions docs/actions/gcp-gsm-load-secrets/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,9 +47,9 @@ To load a secret from GSM figure out the following:

<!-- AUTO-DOC-OUTPUT:START - Do not remove or modify this section -->

| OUTPUT | TYPE | DESCRIPTION |
| ------- | ------ | ---------------------------------- |
| secrets | string | Secrets loaded from Secret Manager |
| OUTPUT | TYPE | DESCRIPTION |
| ------------------ | ------ | ---------------------------- |
| secret-json-string | string | JSON string with all secrets |

<!-- AUTO-DOC-OUTPUT:END -->

Expand Down