Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial implementation #1

Merged
merged 73 commits into from
Jan 20, 2025
Merged
Show file tree
Hide file tree
Changes from 57 commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
4d290a7
Initial implementation
omus Jan 9, 2025
f62fbc8
Update README
omus Jan 13, 2025
47fb313
Add GHAs
omus Jan 14, 2025
dffbe85
Linting pass
omus Jan 14, 2025
146460c
Skip cache-to when pushing is false
omus Jan 14, 2025
7bf6d29
Set default for DEBIAN_VERSION
omus Jan 14, 2025
5da6a0b
Try GHCR
omus Jan 14, 2025
57f3487
Add permissions
omus Jan 14, 2025
15ae11d
Require push
omus Jan 14, 2025
524590b
Overwrite revision annotation
omus Jan 14, 2025
2155516
Fiddle with annotations
omus Jan 14, 2025
0a09220
WIP
omus Jan 14, 2025
fda1a0e
Debug
omus Jan 14, 2025
930cb44
Debug
omus Jan 14, 2025
883d987
Iterating
omus Jan 14, 2025
433a3a1
Iterating
omus Jan 14, 2025
ca0f5d9
Iterating
omus Jan 14, 2025
c5df118
Fix SHA determination
omus Jan 14, 2025
607b10c
Test annotation
omus Jan 14, 2025
557b442
fixup! Test annotation
omus Jan 14, 2025
e89070e
Expand tests
omus Jan 14, 2025
5790c41
Validate build arg
omus Jan 14, 2025
bdb9cd5
WIP
omus Jan 14, 2025
d692d6a
Iterating
omus Jan 14, 2025
f97fce1
Iterating
omus Jan 14, 2025
eccc109
Iterating
omus Jan 14, 2025
ed84975
Unset docker/metadata-action env variables
omus Jan 14, 2025
ecaee3f
Debug
omus Jan 14, 2025
fd930bd
Overwrite docker/metadata-action env
omus Jan 14, 2025
5d7dde0
fixup! Overwrite docker/metadata-action env
omus Jan 14, 2025
d20171e
fixup
omus Jan 14, 2025
3281e83
fixup
omus Jan 14, 2025
a7ed052
Use repository marked as temporary
omus Jan 14, 2025
6769abf
Experiment with GHCR cleanup
omus Jan 14, 2025
686c002
No longer a dry-run
omus Jan 14, 2025
48fecc2
fixup
omus Jan 14, 2025
25c053b
fixup
omus Jan 14, 2025
eee06c9
Test layer caching
omus Jan 14, 2025
f15b05c
fixup! Test layer caching
omus Jan 14, 2025
e788b95
Iterating
omus Jan 14, 2025
6570cb8
fixup! Iterating
omus Jan 14, 2025
0fc558e
fixup! Iterating
omus Jan 14, 2025
e556a7e
Additional details in README
omus Jan 14, 2025
edabbb8
More descriptions
omus Jan 14, 2025
951fe7c
Add comment about test limitations with a single image repo
omus Jan 15, 2025
b4c5d96
Try git context
omus Jan 15, 2025
5921a42
fixup! Try git context
omus Jan 15, 2025
ec612d5
Revert "Try git context"
omus Jan 15, 2025
a22e812
Try an arbitrary SHA
omus Jan 15, 2025
4078dbf
fixup! Try an arbitrary SHA
omus Jan 15, 2025
5925350
Support arbitrary commits
omus Jan 15, 2025
98b26c0
Cleanup overrides
omus Jan 15, 2025
efd8837
Back to restricted SHA
omus Jan 15, 2025
074aa3d
fixup! Cleanup overrides
omus Jan 15, 2025
cb809be
fixup! Cleanup overrides
omus Jan 15, 2025
18bcc6f
Add LICENSE
omus Jan 15, 2025
3a7af71
Update LICENSE year
omus Jan 15, 2025
409084e
Comment on limited commit SHAs
omus Jan 16, 2025
6634dad
drop! Test push trigger
omus Jan 17, 2025
8c0ecfd
Revise tests for push
omus Jan 17, 2025
183adcf
drop! PR event for now
omus Jan 17, 2025
53ca0e4
Test push in PRs
omus Jan 17, 2025
3cf9d0c
Improve test parallelism
omus Jan 20, 2025
9acff75
fixup! Improve test parallelism
omus Jan 20, 2025
ead4c73
fixup! Improve test parallelism
omus Jan 20, 2025
04ffd2f
Remove cache layers during cleanup
omus Jan 20, 2025
bc8228a
Separate PR/push repos
omus Jan 20, 2025
c78889c
Empty: test
omus Jan 20, 2025
cc6e18d
Update concurrency group
omus Jan 20, 2025
57e377b
Add safety check
omus Jan 20, 2025
78dd073
fixup
omus Jan 20, 2025
18b6aaa
Simplify cleanup name
omus Jan 20, 2025
703c6b5
Tag fix
omus Jan 20, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# https://editorconfig.org/

# https://manpages.debian.org/testing/shfmt/shfmt.1.en.html#EXAMPLES
[*.sh]
indent_style = space
indent_size = 4
shell_variant = bash # --language-variant
binary_next_line = false
switch_case_indent = true # --case-indent
space_redirects = false
keep_padding = false
function_next_line = false # --func-next-line
25 changes: 25 additions & 0 deletions .github/workflows/gha.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
---
name: GitHub Actions
on:
pull_request:
paths:
- ".github/workflows/*"

jobs:
lint:
name: Lint
# These permissions are needed to:
# - Checkout the Git repo (`contents: read`)
permissions:
contents: read
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
# https://github.com/rhysd/actionlint/blob/v1.7.6/docs/usage.md#use-actionlint-on-github-actions
# https://github.com/rhysd/actionlint/blob/v1.7.6/docs/usage.md#reviewdog
# https://github.com/reviewdog/reviewdog#filter-mode
# No support for non-workflows yet: https://github.com/rhysd/actionlint/issues/46
- uses: reviewdog/action-actionlint@a1b7ce56be870acfe94b83ce5f6da076aecc6d8c # v1.62.0
with:
fail_level: error
filter_mode: nofilter # Post results on all results and not just changed files
128 changes: 128 additions & 0 deletions .github/workflows/integration-tests.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
---
name: Integration Tests
on:
pull_request:
paths:
- "action.yaml"
- ".github/workflows/integration-tests.yaml"
push:
branches:
- main
paths:
- "action.yaml"
- ".github/workflows/integration-tests.yaml"

jobs:
test:
name: Test ${{ matrix.test.title }}
# These permissions are needed to:
# - Checkout the repo
permissions:
contents: read
packages: write
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
test:
# We need to avoid running concurrent tests using the same commit SHA as
# otherwise we could see tests pass when one of them doesn't say output
# the cache layers. We could address this in two ways in the future if this
# becomes limiting:
#
# 1. Make use of separate image repositories for each test. This allows each
# test to run in parallel without the potential for conflicts.
# 2. Use job concurrency and `max-parallel` for matrix jobs to run jobs
# sequentially with cleanup in between. May be rather slow.
#
# I also considered revising the action to avoid pushing images entirely.
# Doing this may be challenging in otherways as pushing the image is a
# requirement for getting the digests in some contexts:
# https://github.com/docker/build-push-action/issues/906#issuecomment-1674567311
- title: Merge Commit
commit-sha: ${{ github.sha }}
from-scratch: true
- title: Head Commit
commit-sha: ${{ github.event.pull_request.head.sha }}
steps:
- name: Job started at
id: job-started
run: |
job_started_at="$(date --utc --iso-8601=seconds)"
echo "at=$job_started_at" | tee -a "$GITHUB_OUTPUT"
- uses: actions/checkout@v4
with:
ref: ${{ matrix.test.commit-sha }}
- name: Log in to the Container registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ github.token }}
- uses: ./
id: build
with:
image-repository: ghcr.io/beacon-biosignals/temporary/whalesay
context: test
build-args: |
DEBIAN_VERSION=12.9
from-scratch: ${{ matrix.test.from-scratch || 'false' }}
- name: Validate image works
run: |
docker pull "${{ steps.build.outputs.image }}"
output="$(docker run "${{ steps.build.outputs.image }}")"
if [[ "$(wc -l <<<"$output")" -lt 14 ]]; then
echo "$output"
exit 1
fi
debian_version="$(docker run --entrypoint=/bin/cat "${{ steps.build.outputs.image }}" /etc/debian_version)"
[[ "$debian_version" == "12.9" ]] || exit 2
- name: Layer created at
id: layer-created
run: |
layer_created_at="$(docker run --entrypoint=/bin/cat "${{ steps.build.outputs.image }}" /etc/layer-created-at)"
echo "at=$layer_created_at" | tee -a "$GITHUB_OUTPUT"
- name: Validate layer caching
if: ${{ matrix.test.from-scratch == false }}
run: |
[[ "$(date -d "$layer_created_at" +%s)" -lt "$(date -d "$job_started_at" +%s)" ]] || exit 1
env:
job_started_at: ${{ steps.job-started.outputs.at }}
layer_created_at: ${{ steps.layer-created.outputs.at }}
Comment on lines +138 to +144
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will fail on main the first time it is run. I'll just re-run the workflow for that scenario

- name: Validate no layer caching
if: ${{ matrix.test.from-scratch == true }}
run: |
[[ "$(date -d "$layer_created_at" +%s)" -gt "$(date -d "$job_started_at" +%s)" ]] || exit 1
env:
job_started_at: ${{ steps.job-started.outputs.at }}
layer_created_at: ${{ steps.layer-created.outputs.at }}
- name: Validate cache images
run: |
docker manifest inspect "${{ steps.build.outputs.image-repository }}:cache-sha-${{ matrix.test.commit-sha }}"
docker manifest inspect "${{ steps.build.outputs.image-repository }}:cache-branch-${branch//[^[:alnum:]]/-}"
env:
branch: ${{ github.head_ref || (github.ref_type == 'branch' && github.ref_name) }}
- name: Validate annotations
run: |
set -x
json="$(docker manifest inspect "${{ steps.build.outputs.image }}")"
[[ "$(jq -r '.annotations."org.opencontainers.image.revision"' <<<"$json")" == "${{ matrix.test.commit-sha }}" ]] || exit 1
- name: Validate docker/metadata-output environment variables are overwritten
shell: bash
run: |
if [[ "$(printenv | grep '^DOCKER_METADATA_OUTPUT_' | grep -c '[^=]$')" -ne 0 ]]; then
printenv | grep '^DOCKER_METADATA_OUTPUT_'
exit 1
fi

cleanup:
name: Cleanup
needs: test
runs-on: ubuntu-latest
steps:
- uses: dataaxiom/ghcr-cleanup-action@cd0cdb900b5dbf3a6f2cc869f0dbb0b8211f50c4 # v1.0.16
with:
package: temporary/whalesay
older-than: 1 day
keep-n-tagged: 0
exclude-tags: branch-main,cache-branch-main
34 changes: 34 additions & 0 deletions .github/workflows/shell.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
---
name: Shell
on:
pull_request:
paths:
- "**.sh"
- ".github/workflows/*"

jobs:
lint-format:
name: Lint & Format
# These permissions are needed to:
# - Checkout the Git repo (`contents: read`)
# - Post a comments on PRs: https://github.com/luizm/action-sh-checker#secrets
permissions:
contents: read
pull-requests: write
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Extract workflow shell scripts
id: extract
uses: beacon-biosignals/gha-extract-shell-scripts@v1
- uses: luizm/action-sh-checker@c6edb3de93e904488b413636d96c6a56e3ad671a # v0.8.0
env:
GITHUB_TOKEN: ${{ github.token }}
with:
sh_checker_comment: true
# Support investigating linting/formatting errors
- uses: actions/upload-artifact@v4
if: ${{ failure() }}
with:
name: workflow-scripts
path: ${{ steps.extract.outputs.output-dir }}
18 changes: 18 additions & 0 deletions .github/workflows/yaml.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
---
# https://yamllint.readthedocs.io/en/stable/integration.html#integration-with-github-actions
name: YAML
on:
pull_request:
paths:
- "**/*.yaml"
- "**/*.yml"
jobs:
lint:
name: Lint
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install yamllint
run: pip install yamllint
- name: Lint YAML files
run: yamllint . --format=github
8 changes: 8 additions & 0 deletions .yamllint.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
rules:
indentation:
spaces: 2
indent-sequences: true
document-start:
present: true
new-line-at-end-of-file: enable
21 changes: 21 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2025 Beacon Biosignals

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
70 changes: 69 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,69 @@
# docker-build
# Docker Build

Build a Docker image while utilizing [layer caching](https://docs.docker.com/build/cache/) backed from the image repository. Image tags will be automatically created based upon the relevant PR, branch name, and commit SHA.

When using this action we recommend utilizing a separate image repositories for development and production (e.g.`temporary/my-image` and `permanent/my-image`) to make it easier to separate temporary images from permanent images meant for end users. The `beacon-biosignals/docker-build` action is used to build temporary images under development. Once a temporary image is ready for production it can be promoted to be permanent by using `docker tag`/`docker push` or [`regctl image copy --digest-tags`](https://github.com/regclient/regclient/blob/main/docs/regctl.md#registry-commands) (if you want the digest to be identical across registries) to transfer the image.

Note that although [Docker does support using GitHub Actions cache](https://docs.docker.com/build/cache/backends/gha/) as a layer cache backend the GHA cache limit for a repository is 10 GB which is quite limiting for larger Docker images.

## Example

```yaml
---
on:
pull_request: {}
# Trigger this build workflow on "main". See `from-scratch`
push:
branches:
- main
jobs:
example:
# These permissions are needed to:
# - Get the workflow run: https://github.com/beacon-biosignals/docker-build#permissions
permissions: {}
runs-on: ubuntu-latest
steps:
- name: Build image
uses: beacon-biosignals/docker-build@v1
with:
image-repository: temporary/my-image
context: .
# Example of passing in Docker `--build-arg`
build-args: |
JULIA_VERSION=1.10
PYTHON_VERSION=3.10
# Example of passing in Docker `--secret`
build-secrets: |
github-token=${{ secrets.token || github.token }}
# Build images from scratch on "main". Ensures that caching doesn't result in using insecure system packages.
from-scratch: ${{ github.ref == 'refs/heads/main' }}
```

## Inputs

| Name | Description | Required | Example |
|:---------------------|:------------|:---------|:--------|
| `image-repository` | The Docker image repository to push the build image and cached layers. | Yes | `temporary/my-image` |
| `context` | The Docker build context directory. Defaults to `.`. | No | `./my-image` |
| `build-args` | List of [build-time variables](https://docs.docker.com/reference/cli/docker/buildx/build/#build-arg). | No | <pre><code>HTTP_PROXY=http://10.20.30.2:1234&#10;FTP_PROXY=http://40.50.60.5:4567</code></pre> |
| `build-secrets` | List of [secrets](https://docs.docker.com/engine/reference/commandline/buildx_build/#secret) to expose to the build. | No | `GIT_AUTH_TOKEN=mytoken` |
| `from-scratch` | Do not use cache when building the image. Defaults to `false`. | No | `false` |

## Outputs

| Name | Description | Example |
|:-------------------|:------------|:--------|
| `image` | Reference to the build image including the digest. | `temporary/my-image@sha256:37782d4e1c24d8f12047039a0d3512d1b6059e306a80d5b66a1d9ff60247a8cb` |
| `image-repository` | The Docker image repository where the image was pushed to. | `temporary/my-image` |
| `digest` | The built Docker image digest. | `sha256:37782d4e1c24d8f12047039a0d3512d1b6059e306a80d5b66a1d9ff60247a8cb` |
| `tags` | JSON list of tags associated with the built Docker image. | `branch-main`, `sha-152cb14` |
| `commit-sha` | The Git commit SHA used to build the image. | `152cb14643b50529b229930d6124e6bbef48668d` |

## Permissions

The follow [job permissions](https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs) are required to run this action:

```yaml
permissions:
packages: write # Only required when using the GitHub Container registry: https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry
```
Loading
Loading