Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add GitHub Actions workflow to build, test and push Docker images. #96

Merged
merged 2 commits into from
Nov 8, 2023

Conversation

smarnach
Copy link
Contributor

@smarnach smarnach commented Nov 7, 2023

This is a first fully working GitHub Actions workflow to build, test and upload Docker images for Eliot.

The workflow is triggered when a pull request against the main branch is opened, updated or merged, or when a new tag is created.

The workflow consists of two jobs. The build job builds, tests and stores an image as an GitHub Actions artifact. This job is always run. The push job is only run when the main branch is updated (pushing the latest tag) or when a new tag has been created (using the tag as the Docker image tag).

I tested this workflow on a private fork of Eliot. I temporarily gave that fork permission to push to the Eliot GAR repository in the new GCP project. The workflow appears to work fine, and does not leak any GCP project ids in case we want to avoid that.

We could avoid uploading and downloading the Docker image if we would merge the two jobs into a single one. However, this would have the following downsides:

  • The push job has access to the "secrets". While none of the secrets are particularly sensitive, we still don't want to make them available to fork builds. The build job does not have access to any sensitive information, and we could run it on fork PRs.

  • We would need to repeat the if condition for the push job for every single step if we merge the workflows. We could work around this problem by factoring this job out into a reusable action (which we probably want to do anyway, see below).

Problems with the current approach:

  • Artifact storage to store Docker images is slow and feels artificial. It may also be subject to race conditions, since concurrent workflows could overwrite the Docker image before it is used by the push job. We could look into using GitHub Packages to store the Docker image, but that doesn't support any retention policies as far as I can tell, so we'd need to come up with an automated clean-up mechanism.

  • The push job should be made generic, so we (and others) can use it in other repositories.

Overall, I'm leaning towards factoring out the push job into a reusable action and then merging the two jobs into a single one.

@smarnach
Copy link
Contributor Author

smarnach commented Nov 7, 2023

The actions run for this PR failed because we need to get Google's GitHub actions whitelisted for the mozilla-services org. The only error message we get in this case is "Startup failure", which is ludicrously bad.

@smarnach
Copy link
Contributor Author

smarnach commented Nov 8, 2023

I factored pushing to GAR out into an "action" in mozilla-it/deploy-actions#18, and I merged the two workflows into a single one. Once that PR is merged, we can update the reference to the action here, which currently references the feature branch.

Copy link
Contributor

@willkg willkg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything looks ok to me as far as I can tell.

The only curiosity I had was whether it would be better to restrict tags that the workflow kicks off with to v*.

@smarnach
Copy link
Contributor Author

smarnach commented Nov 8, 2023

The only curiosity I had was whether it would be better to restrict tags that the workflow kicks off with to v*.

I don't have a strong opinion here, and I don't know the exact deployment model we will end up with. I think it's ok to create matching Docker tags for each git tag. We can still decide what images we actually want to deploy. We can also change this workflow if we notice this causes problems. For now I'll just merge it as it is.

@smarnach smarnach merged commit be74cd6 into main Nov 8, 2023
1 check passed
@willkg willkg deleted the github-actions branch November 15, 2023 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants