Verifying GitHub specific claims

[jku]

It would be very useful if it was possible to verify the GitHub specific claims in the certificate that we produce when using GitHub Actions. This is because the claims allow verifying that:

• Signing happened on GitHub Actions builder...
• while running the specific workflow...
• using the specified commit SHA..
• of the specified GitHub project

This completely defines the build process in an auditable way.

This issue is related to #106 and #124 but I figured it would make sense to have a specific issue as I have prototype code...

I've said this before but I'll restate: this GitHub-specific functionality does not have to live in sigstore-python but we should make sure it's somehow possible.

[jku]

See https://github.com/jku/sigstore-python/commits/github-verify for a prototype. It's a bit rough but I'll open a draft PR one of these days...

In my branch I've included the certificate in the VerificationSuccess result: this could be made private but as public would also make it possible to implement this kind of functionality outside of the sigstore-python codebase, assuming Verifier is public API.

Usage:

python -m sigstore verify-github \
    --repository sigstore/sigstore-python \
    --workflow .github/workflows/release.yml \
    --tag v0.5.1 \
    --sha 05b7ab4de650271873469ef087273c28d1cac4d7 \
    sigstore-0.5.1-py3-none-any.whl

tag and sha are optional, others are required.

The API is not quite final - e.g. repository currently defines both work flow repository and source repository. This makes sense for the common case but there are probably cases where an override is needed.

[di]

Agreed that we should allow users to verify any information that's available and not artificially prevent them from doing so. I do have concerns about this becoming unwieldy given the number of claims/args we'd need to support (multiplied by all the different OIDC providers) so I'd definitely want to think about this in the context of #124 and with a thought towards how we can make this easily generalizable (i.e., not require changes to the CLI if we introduce a new supported OIDC provider with different claims).

[woodruffw]

Agreed with #157 (comment). One thing I'm particularly afraid of is a gpg-style explosion of flags with ambiguous semantics/context-sensitivity (not that any of the ones you're proposing are ambiguous!)

[jku]

I do have concerns about this becoming unwieldy given the number of claims/args we'd need to support (multiplied by all the different OIDC providers) so I'd definitely want to think about this in the context of #124 and with a thought towards how we can make this easily generalizable

Yeah, it is a very valid concern and the main reason for not just creating a PR... I've played with the configuration format idea as well and see two problems with it:

• To solve the problem in this issue, you still need to provide an easy and safe way to define what claims should be verified for the GitHub case out-of-band (as in, the config file can't be downloaded with the certs and signatures). I believe you still end up creating a script that produces a known good configuration file for GitHub (and then another script for another provider). If you end up creating a script, why not just call the Verifier API from the script directly?
• the configuration easily becomes quite a bit more verbose and difficult to understand than my CLI example is (because the CLI is customized to best fit the specific use case) and even then it could be difficult to make it generic enough to cover everything that could be inserted into cert extensions

A potential path forward is to make the Verifier API generally useful, and implement the provider specific tools as separate from the main sigstore module -- in other modules/binaries or outside the project even

[jku]

So I've come back to this after a bit of a break and I think this is still the state of things:

• It's currently not possible to verify anything useful about the claims that GitHub makes in their certificate (the only thing sigstore-python can verify is that the issuer is GitHub): If I want to verify anything else I need to start parsing the certificate file myself
• The idea of adding GitHub-specific functionality to sigstore-python has received appropriate levels of scepticism here :) I admit I agree with this criticism
• sigstore-python API currently does not make it easy to create this sort of functionality elsewhere

I think the last item could be fairly easy to fix. I'll open a draft PR to get some opinions on this approach.

[jku]

@di ^ this is what I was referring to in earlier discussion.

The context here is a "Python Packaging & Release Using Continuous Deployment" tutorial we're planning where I would really like to use sigstore-python to sign the release on GitHub... but it feels wrong to recommend signing if it's still impossible to verify the signature (in a meaningful way): that's just cargo cult signing.

I will plug sigstore-python as the future tool in any case, but if I could show how to actually verify these GitHub signatures (even with an external script shown in my PR comment) I think we'd include sigstore sign ... in the tutorial itself.

[di]

Yes, I think we're blocked on some movement on sigstore/cosign#1964 so we can resolve #108 and have a way to verify arbitrary extensions (without needing to provide some GitHub-specific mechanism).

[jku]

The --cert-email/--cert-identity issue is clearly solvable... but I'm not sure exposing arbitrary extensions can lead to acceptable UX (whether the extensions are exposed to user via a command line interface or a configuration file that the verifier writes)? If it can be successfully done then that would certainly solve the problem.

Some issues I've seen with exposing extensions (examples for GitHub obviously):

• readability is pretty awful: --check-extension 1.3.6.1.4.1.57264.1.3 bff2e635da74627b62b31efd746f533bf97801ed -- it's quite hard to see if that is the correct check or not
• logical input from user is items like repository and tag, but what certificate actually contains is e.g. URI:https://github.com/sigstore/sigstore-python/.github/workflows/release.yml@refs/tags/v0.6.3 -- requiring that from a user seems a bit much when we can easily do better
• it's beneficial to be able to choose how to validate based in input: e.g. if tag is not given as input then checking the SAN prefix is still useful as it fully verifies which GitHub project this is from -- this is hard to express without code

[haydentherapper]

FYI this was added in sigstore/cosign#1626 for Cosign. We went with individual flags for each GitHub claim.

[haydentherapper]

I think the UX isn't ideal, and that the long-term solution would be a policy file. For one-off verifications, I think flags are fine.

[woodruffw]

I'm going to include the backing work for this in #250.

[woodruffw]

The API is done here, so I'm closing this in favor of #322 for tracking the CLI support.