Initial implementation

[omus]

TODO:

• Discuss if GHCR should be used for these tests (we do need an image repository for the tests)
• Make the GHCR image repository have public visibility

[github-actions]

sh-checker report

To get the full details, please check in the job output.

shellcheck errors

'shellcheck ' returned error 1 finding the following syntactical issues:

----------

In workflow_scripts/integration-tests.yaml/job=Test_{{_matrix.test.title_}}/step=Validate_docker-metadata-output_environment_variables_are_overwritten.sh line 4:
if [[ "$(printenv | grep '^DOCKER_METADATA_OUTPUT_' | grep '[^=]$' | wc -l)" -ne 0 ]]; then
                                                      ^----------^ SC2126 (style): Consider using 'grep -c' instead of 'grep|wc -l'.

For more information:
  https://www.shellcheck.net/wiki/SC2126 -- Consider using 'grep -c' instead ...
----------

You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
  # shellcheck disable=NNNN
above the line that contains the issue, where NNNN is the error code;
3. Add '-e NNNN' to the SHELLCHECK_OPTS setting in your .yml action file.

shfmt errors

'shfmt ' found no issues.

[kleinschmidt]

one more time to check my understanding: this is meant to run on checked out code. when it's run in a workflow that's triggered by a PR, because this workflow doesn't do the checking out, we need to be able to handle both the case where the checked out code is hte merge commit and the case where it's the head commit, is that right? could you add a comment to explain that?

[omus]

The Docker image we build is intended to use a directory base context. With this the image which is built is based upon the commit used in actions/checkout. I intended to just simply do git rev-parse HEAD to get the checked out commit but the issue is that the docker/metadata-action when generating tags and annotations will end up pulling details about the commit SHA from the workflow metadata. This is why we need to use DOCKER_METADATA_PR_HEAD_SHA at all to instruct the action which SHA to use when running as a PR.

I did some experimenting here and tried to create the tags more manually using type=raw but ran into issue where the docker/metadata-action would fail when trying to use an arbitrary commit. As this was turning into a rabbit hole I didn't want to keep going down I opted to limit this action to only support PR merge commits and the head commit which are only use cases we use internally.

The primary point of concern is we don't want to have our image tags or annotations stating the were built with one commit SHA when in reality they were built with another.

I'll open an issue for this.

[omus]

#2

[omus]

I'm going to get some other work done and then come back to these tests. I think I'd like to test all of these in parallel:

• PR with merge commit
• PR with head commit
• Push with head commit

Getting this to work in a matrix is weird so I may need to deal with the limiting parallel testing factors or consider some other options.

[github-actions]

sh-checker report

To get the full details, please check in the job output.

shellcheck errors

'shellcheck ' returned error 1 finding the following syntactical issues:

----------

In workflow_scripts/integration-tests.yaml/job=Filter_Matrix/step=Filter_Matrix.sh line 26:
yq -o=json <<<"$output_yaml" | jq -e '(map({package, "commit-sha"}) | unique | length) == length' || exit 1
               ^----------^ SC2154 (warning): output_yaml is referenced but not assigned.

For more information:
  https://www.shellcheck.net/wiki/SC2154 -- output_yaml is referenced but not...
----------

You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
  # shellcheck disable=NNNN
above the line that contains the issue, where NNNN is the error code;
3. Add '-e NNNN' to the SHELLCHECK_OPTS setting in your .yml action file.

shfmt errors

'shfmt ' found no issues.

[omus]

I've revised the tests fairly heavily to allow for more parallelism and testing on both push and pull_request events.