Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to access the built image sha #49

Open
dhensby opened this issue Mar 1, 2024 · 22 comments · Fixed by #59
Open

How to access the built image sha #49

dhensby opened this issue Mar 1, 2024 · 22 comments · Fixed by #59
Labels
feature request question Further information is requested

Comments

@dhensby
Copy link

dhensby commented Mar 1, 2024

I'm looking at implementing docker image signing as part of our release process. To do this, ideally we'd have access to the sha of the built image. Is there any way to expose this value so that it can be used in an exec step or similar?

@esatterwhite
Copy link
Owner

esatterwhite commented Mar 13, 2024

@dhensby
Currently, I don't think so. I don't think its possible to pass information between stages, or at least it isn't a documented/supported feature of semantic-release.

How would you want to use this? an example would be helpful.

@esatterwhite esatterwhite added the question Further information is requested label Mar 13, 2024
@dhensby
Copy link
Author

dhensby commented Mar 13, 2024

So using something like notion you (ideally) want to get your hands on the raw image sha for signing. It's not much more than: notation sign --signature-format cose $IMAGE (where $IMAGE is image@sha:xxxx see: https://notaryproject.dev/docs/quickstart-guides/quickstart-sign-image-artifact/)

The issue is that builds don't seem to have a predictable tag nor a way to get your hand on it so you can resolve the hash locally.

Though, looking at the code - perhaps if I can set build_id, instead of it being random, the local tag will be predictable and I'll be able to find the image 🤔

@dhensby
Copy link
Author

dhensby commented Mar 13, 2024

If there was a way to set this build_id deterministically at run time, then I'd be able to run: docker images --no-trunc --quiet "image-name:${build_id}" to get the hash

@esatterwhite
Copy link
Owner

If there was a way to set this build_id deterministically at run time, then I'd be able to run: docker images --no-trunc --quiet "image-name:${build_id}" to get the hash

Not currently, but it wouldn't be difficult to add.

Also, and I'll have to confirm, but I might just be able to mutate the nextRelease object with the docker build info. Then you could reference it other states / plugins provided they support templating of some kind

@esatterwhite
Copy link
Owner

Additionally as I'm working through it now, I'm not sure that would work if you were using buildx as the build + push happen in the same docker command and the images are not stored locally.

it may not effect you, but something to think about.

@dhensby
Copy link
Author

dhensby commented Mar 13, 2024

The images are available locally if the clean flag is set to false, right? 🤔

@esatterwhite
Copy link
Owner

esatterwhite commented Mar 13, 2024

This images are available locally if the clean flag is set to false, right? 🤔

Yes.
I'm talking in a future state, if you were to start building multi-arch images w/ buildx ( a thing I'm working on now ), that would not be the case.

So nothing to worry about here, I'm just thinking out loud.

@Th3S4mur41
Copy link

Looking for something similar in order to take advantage of actions/attest-build-provenance in GitHub CI.

Right now, this seems only to be possible by publishing the docker image using docker/build-push-action after semantic-release.

@esatterwhite
Copy link
Owner

How would you expect to get access to it?

@esatterwhite
Copy link
Owner

The final image sha, or the tag of an image?

@dhensby
Copy link
Author

dhensby commented Sep 2, 2024

The signing programs can complain if you don't give the actual image sha, so that would be idea.

@Th3S4mur41
Copy link

How would you expect to get access to it?

I'm not sure either...
In order to get the version, I extract it from the package.json and add it as an output parameter for the CI job.

echo "version=$(npm run env | grep npm_package_version | cut -d '=' -f 2)" >> $GITHUB_OUTPUT

I can then reuse the new version in any of the following jobs.
If there is any way I could access the sha of the published image, I could do something similar

@dhensby
Copy link
Author

dhensby commented Sep 3, 2024

At the moment I use the exec provider for semantic release to add the release version as an output:

  - ['@semantic-release/exec', {
      "publishCmd": 'echo "version=${nextRelease.version}" >> "$GITHUB_OUTPUT"'
    }]

Then I use that version to do the signing with notation: ./notation sign --signature-format cose --id "$CERT_ID" --plugin azure-kv --plugin-config self_signed=true "$IMAGE"

Where $IMAGE is and env var constructed as so: dockerhost.com/image-name:${{ steps.release.outputs.version }}


Edit (for clarity): This causes notation to complain because it wants a sha instead - and that's what I'd like to be able to get my hands on somehow even if it means another exec call that just adds the image sha to the output.

@esatterwhite
Copy link
Owner

Ah, I see. Thanks, that helps. I think mutating the release object could work. Provided semantic release isn't re-creating it for each plugin

@esatterwhite
Copy link
Owner

esatterwhite commented Dec 23, 2024

@dhensby @Th3S4mur41 The idea of just tacking something on to the plugin context doesn't seem feasible. The context appears to be sandboxed to a plugin. I can attach information and that is visible to that plugin's stages, but the context is missing that information from other plugins even at later stages.

@esatterwhite
Copy link
Owner

@dhensby @Th3S4mur41
If this is mostly a github actions thing. I don't see a problem with adding support for github actions. I could set some image information in actions outputs

esatterwhite added a commit that referenced this issue Dec 26, 2024
There have been several requests to gain access or over ride the build
id of the image so that the caller can control that and reference it
later in a build process. Initial attempts to expose this information
from the nextRelease property. Unfortunately, this object seems to be
sandboxed, and additional information is not carried to other plugins.
As an alternative, this adds basic support for github actions be setting
output and environment variable that include the build id and image sha
where possible

fixes: #49
@esatterwhite esatterwhite linked a pull request Dec 26, 2024 that will close this issue
@esatterwhite
Copy link
Owner

@dhensby @Th3S4mur41 take a look at #59 and let me know if this is helpful

@Th3S4mur41
Copy link

Th3S4mur41 commented Dec 29, 2024

Thanks @esatterwhite this does looks promising and should solve my issue.
I'll try it out as soon as it gets released

esatterwhite added a commit that referenced this issue Dec 30, 2024
There have been several requests to gain access or over ride the build
id of the image so that the caller can control that and reference it
later in a build process. Initial attempts to expose this information
from the nextRelease property. Unfortunately, this object seems to be
sandboxed, and additional information is not carried to other plugins.
As an alternative, this adds basic support for github actions be setting
output and environment variable that include the build id and image sha
where possible

fixes: #49
@Th3S4mur41
Copy link

@esatterwhite Just tested it...
While I was able to read the values from the output of the release, the SEMANTIC_RELEASE_DOCKER_IMAGE_SHA_LONG doesn't seem to be the correct image digest

Looking at the manifest, the SEMANTIC_RELEASE_DOCKER_IMAGE_SHA_LONG seems to be the config digest (green), while the attestation job actually requires the image digest (pink)

image

image

@esatterwhite
Copy link
Owner

esatterwhite commented Jan 2, 2025

Interesting. What is the sha reported by docker inspect <image>
which one is that pointing to?
@Th3S4mur41

@Th3S4mur41
Copy link

The ID is the one returned in SEMANTIC_RELEASE_DOCKER_IMAGE_SHA_LONG

For the attestation, the sha from RepoDigests would be necessary though.

docker inspect ghcr.io/th3s4mur41/hw2energyid
[
    {
        "Id": "sha256:84d78f623c6ea36b8111fee0edc21df2979cdb0974788df6bd2ea94e15bc6d29",
        "RepoTags": [
            "ghcr.io/th3s4mur41/hw2energyid:latest"
        ],
        "RepoDigests": [
            "ghcr.io/th3s4mur41/hw2energyid@sha256:9efe6f2c9877cccdc7d91cc53a568e928fd87168cb71d85b288b2216578a7d10"
        ],
        "Parent": "",
        "Comment": "buildkit.dockerfile.v0",
        "Created": "2025-01-02T11:13:50.735252468Z",
        "Container": "",
        "ContainerConfig": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": null,
            "Cmd": null,
            "Image": "",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": null
        },
        "DockerVersion": "",
        "Author": "",
        "Config": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "NODE_VERSION=22.12.0",
                "YARN_VERSION=1.22.22",
                "energyid=",
                "p1=",
                "meter="
            ],
            "Cmd": [
                "/bin/sh",
                "-c",
                "npx hw2energyid --energyid=${energyid} --meter=${meter} -r"
            ],
            "ArgsEscaped": true,
            "Image": "",
            "Volumes": null,
            "WorkingDir": "/app",
            "Entrypoint": [
                "docker-entrypoint.sh"
            ],
            "OnBuild": null,
            "Labels": null
        },
        "Architecture": "amd64",
        "Os": "linux",
        "Size": 425865537,
        "VirtualSize": 425865537,
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/5184c99c1b04f292ffe2fdc096512cc87895809a24994e181cfbbfec6c8fa00a/diff:/var/lib/docker/overlay2/002e8890f0473177e09637e55d758ba07bf5916d580378539e51656ffe686340/diff:/var/lib/docker/overlay2/a0879996233c1a1964e82aab54dd5eeabe35134bc01b127cd8e61e190b53e516/diff:/var/lib/docker/overlay2/edb8cc4cd92c6613e028245be7ff2546e631d844a0315bdb21471d18faa177ea/diff:/var/lib/docker/overlay2/e05f26de2d2301087129bdea6f3b56d23a0cb109da5212b3246fd961ef0876ad/diff:/var/lib/docker/overlay2/10a8443be5f27a0b753abdbab7d993c5aec91788625badb29bb395ac83b8f8ee/diff:/var/lib/docker/overlay2/bfa7aa9463dc9ea97404f40caa7f50f7eef54e9eed28d227d8a9c26d4cc1f6ea/diff:/var/lib/docker/overlay2/550730c0948b6f69977ce11cc002a11f8c9296704ee27fe335f7d08d8dcfbe68/diff:/var/lib/docker/overlay2/7f3c72454c1871cf96982f063b7a241e1da231d50846640abc519dc58b2a91f1/diff:/var/lib/docker/overlay2/8873f576d4fb5548096290a1789bdb9c0ae8827f20b588f942722df2c4e1c530/diff",
                "MergedDir": "/var/lib/docker/overlay2/27e80a7fd71f429a63bc1efabbd9504deb92861e1c0a39a45891958c32bb1615/merged",   
                "UpperDir": "/var/lib/docker/overlay2/27e80a7fd71f429a63bc1efabbd9504deb92861e1c0a39a45891958c32bb1615/diff",      
                "WorkDir": "/var/lib/docker/overlay2/27e80a7fd71f429a63bc1efabbd9504deb92861e1c0a39a45891958c32bb1615/work"        
            },
            "Name": "overlay2"
        },
        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:8b296f48696071aafb5a6286ca60d441a7e559b192fc7f94bb63ee93dae98f17",
                "sha256:2978e8b93e850e57bcd042dae07529cc63a5c86ee67734a98d0e824f24065316",
                "sha256:a6426a9bf417c58d5ba0cf60167becc42891cc011b37f37af53dd3c681a7c08d",
                "sha256:a2171f1263ef56c78fc4a5d5e18df026b82821a968962204a236dbca102bdc2d",
                "sha256:b77d99c4c05e0efa397eb96032db3efb9d5f4660f476dc8a4211d25a021776ac",
                "sha256:bc6e99bef67af7103a35855b5c684f4eaae91196f889ddc81554938904b8a47c",
                "sha256:1f404b0c262b6b8907365a418530440e98752e58e5e5507a4e30eaec3b98b3a6",
                "sha256:ad4fc2c519bc7efb9e95ce409c62e5af86f34692e22e2993dbf731a97d5ad422",
                "sha256:3eb25a9d57f8d36875e1e708dee07281e38684baef1e359177dd096a779e5834",
                "sha256:91fee93709b72e68fc491281a387fd1259077f50a76febbf12ed6566bbfcd951",
                "sha256:d4599d0ff2e40be18e11bb35e867c4916ed341db8f52799e9ec8261447efee29"
            ]
        },
        "Metadata": {
            "LastTagTime": "0001-01-01T00:00:00Z"
        }
    }
]

Looking at the release logs in the job, both seem to be available:

image
image

@esatterwhite
Copy link
Owner

esatterwhite commented Jan 2, 2025

Yes, its current looking for the pattern ^sha<number>:<alphanumeric>$

I think instead of trying to parse through all the output would be to try pull the manifest after the push and extra the appropriate hash values from the metadata.

That would also work w/ buildx and I could construct additional environment variables per platform if necessary.

@esatterwhite esatterwhite reopened this Jan 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request question Further information is requested
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants