Offline Rekor bundle generation and verification (#247)

* _cli: flag scaffolding for offline rekor verification Signed-off-by: William Woodruff <[email protected]> * _cli: more scaffolding Signed-off-by: William Woodruff <[email protected]> * sigstore: refactor RekorEntry/SET verification for offline bundles Signed-off-by: William Woodruff <[email protected]> * _cli: add envvar defaults for new flags Signed-off-by: William Woodruff <[email protected]> * README: update `sigstore verify --help` Signed-off-by: William Woodruff <[email protected]> * _cli: handle `verify --offline` correctly Signed-off-by: William Woodruff <[email protected]> * rekor/client: fix docstring The returned value here is not base64-encoded. Signed-off-by: William Woodruff <[email protected]> * _cli: Add `rekor` suffix to offline bundle flags/options Signed-off-by: William Woodruff <[email protected]> * README: update `sigstore verify` Signed-off-by: William Woodruff <[email protected]> * _verify: elaborate on the properties of a non-inclusion-proof verification Signed-off-by: William Woodruff <[email protected]> * _verify: fix comment typos, reflow comments Signed-off-by: William Woodruff <[email protected]> * Apply suggestions from code review Co-authored-by: Dustin Ingram <[email protected]> Signed-off-by: William Woodruff <[email protected]> * _cli: lint Signed-off-by: William Woodruff <[email protected]> * rekor/client: fix capitalization on Payload key Signed-off-by: William Woodruff <[email protected]> * rekor/client: fix keys Signed-off-by: William Woodruff <[email protected]> * _cli: --rekor-bundle implies --rekor-offline In other words: if a user explicitly passes a bundle filename, we never fall back on online verification. Signed-off-by: William Woodruff <[email protected]> * sigstore, test: create and use a separate RekorBundle model This makes validation a little simpler. Signed-off-by: William Woodruff <[email protected]> * sigstore, test: add offline bundle generation Signed-off-by: William Woodruff <[email protected]> * sigstore: blacken Signed-off-by: William Woodruff <[email protected]> * test: add an offline rekor test Signed-off-by: William Woodruff <[email protected]> * _cli: tweak `--rekor-offline` language slightly To emphasize that the absence of `--rekor-offline` does not always imply fully online verification. Signed-off-by: William Woodruff <[email protected]> * README: update `--help` blocks Signed-off-by: William Woodruff <[email protected]> * test: unused import Signed-off-by: William Woodruff <[email protected]> * sigstore: test Rekor entry's consistency against signing artifacts Signed-off-by: William Woodruff <[email protected]> * conftest: strip trailing whitespace from cert and sig Trailing whitespace from the signature was breaking the Rekor consistency check. Signed-off-by: William Woodruff <[email protected]> * treewide: use .rekor for offline rekor bundle files Signed-off-by: William Woodruff <[email protected]> * _verify: lint fixes Signed-off-by: William Woodruff <[email protected]> * _verify: more lint fixes Signed-off-by: William Woodruff <[email protected]> * README, _cli: `--rekor-offline` -> `--require-rekor-offline` Signed-off-by: William Woodruff <[email protected]> * Apply suggestions from code review Co-authored-by: Hayden B <[email protected]> Signed-off-by: William Woodruff <[email protected]> * _verify: clarify comments, add a long comment explaining process Signed-off-by: William Woodruff <[email protected]> * _verify: blacken Signed-off-by: William Woodruff <[email protected]> Signed-off-by: William Woodruff <[email protected]> Signed-off-by: William Woodruff <[email protected]> Co-authored-by: Dustin Ingram <[email protected]> Co-authored-by: Hayden B <[email protected]>
sigstore · Nov 2, 2022 · 1730a99 · 1730a99
1 parent 230d9dc
commit 1730a99
Show file tree

Hide file tree

Showing 15 changed files with 471 additions and 86 deletions.
diff --git a/.gitignore b/.gitignore
@@ -15,6 +15,7 @@ build
 *.pem
 *.sh
 *.pub
+*.rekor
 
 # Don't ignore these files when we intend to include them
 !sigstore/_store/*.crt
@@ -23,3 +24,4 @@ build
 !test/assets/*.txt
 !test/assets/*.crt
 !test/assets/*.sig
+!test/assets/*.rekor
diff --git a/README.md b/README.md
@@ -89,9 +89,9 @@ usage: sigstore sign [-h] [--identity-token TOKEN] [--oidc-client-id ID]
                      [--oidc-client-secret SECRET]
                      [--oidc-disable-ambient-providers] [--oidc-issuer URL]
                      [--no-default-files] [--signature FILE]
-                     [--certificate FILE] [--overwrite] [--staging]
-                     [--rekor-url URL] [--fulcio-url URL] [--ctfe FILE]
-                     [--rekor-root-pubkey FILE]
+                     [--certificate FILE] [--rekor-bundle FILE] [--overwrite]
+                     [--staging] [--rekor-url URL] [--fulcio-url URL]
+                     [--ctfe FILE] [--rekor-root-pubkey FILE]
                      FILE [FILE ...]
 
 positional arguments:
@@ -115,14 +115,18 @@ OpenID Connect options:
                         --staging) (default: https://oauth2.sigstore.dev/auth)
 
 Output options:
-  --no-default-files    Don't emit the default output files ({input}.sig and
-                        {input}.crt) (default: False)
+  --no-default-files    Don't emit the default output files ({input}.sig,
+                        {input}.crt, {input}.rekor) (default: False)
   --signature FILE, --output-signature FILE
                         Write a single signature to the given file; does not
                         work with multiple input files (default: None)
   --certificate FILE, --output-certificate FILE
                         Write a single certificate to the given file; does not
                         work with multiple input files (default: None)
+  --rekor-bundle FILE, --output-rekor-bundle FILE
+                        Write a single offline Rekor bundle to the given file;
+                        does not work with multiple input files (default:
+                        None)
   --overwrite           Overwrite preexisting signature and certificate
                         outputs, if present (default: False)
 
@@ -147,7 +151,8 @@ Verifying:
 <!-- @begin-sigstore-verify-help@ -->
 ```
 usage: sigstore verify [-h] [--certificate FILE] [--signature FILE]
-                       [--cert-email EMAIL] [--cert-oidc-issuer URL]
+                       [--rekor-bundle FILE] [--cert-email EMAIL]
+                       [--cert-oidc-issuer URL] [--require-rekor-offline]
                        [--staging] [--rekor-url URL]
                        FILE [FILE ...]
 
@@ -163,13 +168,18 @@ Verification inputs:
                         used with multiple inputs (default: None)
   --signature FILE      The signature to verify against; not used with
                         multiple inputs (default: None)
+  --rekor-bundle FILE   The offline Rekor bundle to verify with; not used with
+                        multiple inputs (default: None)
 
 Extended verification options:
   --cert-email EMAIL    The email address to check for in the certificate's
                         Subject Alternative Name (default: None)
   --cert-oidc-issuer URL
                         The OIDC issuer URL to check for in the certificate's
                         OIDC issuer extension (default: None)
+  --require-rekor-offline
+                        Require offline Rekor verification with a bundle;
+                        implied by --rekor-bundle (default: False)
 
 Sigstore instance options:
   --staging             Use sigstore's staging instances, instead of the

diff --git a/sigstore/_cli.py b/sigstore/_cli.py
@@ -34,7 +34,12 @@
     STAGING_OAUTH_ISSUER,
     get_identity_token,
 )
-from sigstore._internal.rekor.client import DEFAULT_REKOR_URL, RekorClient
+from sigstore._internal.rekor.client import (
+    DEFAULT_REKOR_URL,
+    RekorBundle,
+    RekorClient,
+    RekorEntry,
+)
 from sigstore._sign import Signer
 from sigstore._utils import load_pem_public_key
 from sigstore._verify import (
@@ -164,7 +169,7 @@ def _parser() -> argparse.ArgumentParser:
         "--no-default-files",
         action="store_true",
         default=_boolify_env("SIGSTORE_NO_DEFAULT_FILES"),
-        help="Don't emit the default output files ({input}.sig and {input}.crt)",
+        help="Don't emit the default output files ({input}.sig, {input}.crt, {input}.rekor)",
     )
     output_options.add_argument(
         "--signature",
@@ -186,6 +191,17 @@ def _parser() -> argparse.ArgumentParser:
             "Write a single certificate to the given file; does not work with multiple input files"
         ),
     )
+    output_options.add_argument(
+        "--rekor-bundle",
+        "--output-rekor-bundle",
+        metavar="FILE",
+        type=Path,
+        default=os.getenv("SIGSTORE_OUTPUT_BUNDLE"),
+        help=(
+            "Write a single offline Rekor bundle to the given file; does not work with "
+            "multiple input files"
+        ),
+    )
     output_options.add_argument(
         "--overwrite",
         action="store_true",
@@ -247,6 +263,13 @@ def _parser() -> argparse.ArgumentParser:
         default=os.getenv("SIGSTORE_SIGNATURE"),
         help="The signature to verify against; not used with multiple inputs",
     )
+    input_options.add_argument(
+        "--rekor-bundle",
+        metavar="FILE",
+        type=Path,
+        default=os.getenv("SIGSTORE_REKOR_BUNDLE"),
+        help="The offline Rekor bundle to verify with; not used with multiple inputs",
+    )
 
     verification_options = verify.add_argument_group("Extended verification options")
     verification_options.add_argument(
@@ -263,6 +286,12 @@ def _parser() -> argparse.ArgumentParser:
         default=os.getenv("SIGSTORE_CERT_OIDC_ISSUER"),
         help="The OIDC issuer URL to check for in the certificate's OIDC issuer extension",
     )
+    verification_options.add_argument(
+        "--require-rekor-offline",
+        action="store_true",
+        default=_boolify_env("SIGSTORE_REQUIRE_REKOR_OFFLINE"),
+        help="Require offline Rekor verification with a bundle; implied by --rekor-bundle",
+    )
 
     instance_options = verify.add_argument_group("Sigstore instance options")
     _add_shared_instance_options(instance_options)
@@ -308,20 +337,32 @@ def main() -> None:
 
 
 def _sign(args: argparse.Namespace) -> None:
-    # `--no-default-files` has no effect on `--{signature,certificate}`, but we
+    # `--rekor-bundle` is a temporary option, pending stabilization of the
+    # Sigstore bundle format.
+    if args.rekor_bundle:
+        logger.warning(
+            "--rekor-bundle is a temporary format, and will be removed in an "
+            "upcoming release of sigstore-python in favor of Sigstore-style bundles"
+        )
+
+    # `--no-default-files` has no effect on `--{signature,certificate,rekor-bundle}`, but we
     # forbid it because it indicates user confusion.
-    if args.no_default_files and (args.signature or args.certificate):
+    if args.no_default_files and (
+        args.signature or args.certificate or args.rekor_bundle
+    ):
         args._parser.error(
-            "--no-default-files may not be combined with --signature or "
-            "--certificate",
+            "--no-default-files may not be combined with --signature, "
+            "--certificate, or --rekor-bundle",
         )
 
     # Fail if `--signature` or `--certificate` is specified *and* we have more
     # than one input.
-    if (args.signature or args.certificate) and len(args.files) > 1:
+    if (args.signature or args.certificate or args.rekor_bundle) and len(
+        args.files
+    ) > 1:
         args._parser.error(
-            "Error: --signature and --certificate can't be used with explicit "
-            "outputs for multiple inputs",
+            "Error: --signature, --certificate, and --rekor-bundle can't be used "
+            "with explicit outputs for multiple inputs",
         )
 
     # Build up the map of inputs -> outputs ahead of any signing operations,
@@ -331,25 +372,28 @@ def _sign(args: argparse.Namespace) -> None:
         if not file.is_file():
             args._parser.error(f"Input must be a file: {file}")
 
-        sig, cert = args.signature, args.certificate
-        if not sig and not cert and not args.no_default_files:
+        sig, cert, bundle = args.signature, args.certificate, args.rekor_bundle
+        if not sig and not cert and not bundle and not args.no_default_files:
             sig = file.parent / f"{file.name}.sig"
             cert = file.parent / f"{file.name}.crt"
+            bundle = file.parent / f"{file.name}.rekor"
 
         if not args.overwrite:
             extants = []
             if sig and sig.exists():
                 extants.append(str(sig))
             if cert and cert.exists():
                 extants.append(str(cert))
+            if bundle and bundle.exists():
+                extants.append(str(bundle))
 
             if extants:
                 args._parser.error(
                     "Refusing to overwrite outputs without --overwrite: "
                     f"{', '.join(extants)}"
                 )
 
-        output_map[file] = {"cert": cert, "sig": sig}
+        output_map[file] = {"cert": cert, "sig": sig, "bundle": bundle}
 
     # Select the signer to use.
     if args.staging:
@@ -396,20 +440,41 @@ def _sign(args: argparse.Namespace) -> None:
             sig_output = sys.stdout
 
         print(result.b64_signature, file=sig_output)
-        if outputs["sig"]:
-            print(f"Signature written to file {outputs['sig']}")
+        if outputs["sig"] is not None:
+            print(f"Signature written to {outputs['sig']}")
 
         if outputs["cert"] is not None:
-            cert_output = open(outputs["cert"], "w")
-            print(result.cert_pem, file=cert_output)
-            print(f"Certificate written to file {outputs['cert']}")
+            with outputs["cert"].open(mode="w") as io:
+                print(result.cert_pem, file=io)
+            print(f"Certificate written to {outputs['cert']}")
+
+        if outputs["bundle"] is not None:
+            with outputs["bundle"].open(mode="w") as io:
+                bundle = result.log_entry.to_bundle()
+                print(bundle.json(by_alias=True), file=io)
+            print(f"Rekor bundle written to {outputs['bundle']}")
 
 
 def _verify(args: argparse.Namespace) -> None:
-    # Fail if `--certificate` or `--signature` is specified and we have more than one input.
-    if (args.certificate or args.signature) and len(args.files) > 1:
+    # `--rekor-bundle` is a temporary option, pending stabilization of the
+    # Sigstore bundle format.
+    if args.rekor_bundle:
+        logger.warning(
+            "--rekor-bundle is a temporary format, and will be removed in an "
+            "upcoming release of sigstore-python in favor of Sigstore-style bundles"
+        )
+
+    # The presence of --rekor-bundle implies --require-rekor-offline.
+    args.require_rekor_offline = args.require_rekor_offline or args.rekor_bundle
+
+    # Fail if --certificate, --signature, or --rekor-bundle is specified and we
+    # have more than one input.
+    if (args.certificate or args.signature or args.rekor_bundle) and len(
+        args.files
+    ) > 1:
         args._parser.error(
-            "--certificate and --signature can only be used with a single input file"
+            "--certificate, --signature, and --rekor-bundle can only be used "
+            "with a single input file"
         )
 
     # The converse of `sign`: we build up an expected input map and check
@@ -419,24 +484,31 @@ def _verify(args: argparse.Namespace) -> None:
         if not file.is_file():
             args._parser.error(f"Input must be a file: {file}")
 
-        sig, cert = args.signature, args.certificate
+        sig, cert, bundle = args.signature, args.certificate, args.rekor_bundle
         if sig is None:
             sig = file.parent / f"{file.name}.sig"
         if cert is None:
             cert = file.parent / f"{file.name}.crt"
+        if bundle is None:
+            bundle = file.parent / f"{file.name}.rekor"
 
         missing = []
         if not sig.is_file():
             missing.append(str(sig))
         if not cert.is_file():
             missing.append(str(cert))
+        if not bundle.is_file() and args.require_rekor_offline:
+            # NOTE: We only produce errors on missing bundle files
+            # if the user has explicitly requested offline-only verification.
+            # Otherwise, we fall back on online verification.
+            missing.append(str(bundle))
 
         if missing:
             args._parser.error(
                 f"Missing verification materials for {(file)}: {', '.join(missing)}"
             )
 
-        input_map[file] = {"cert": cert, "sig": sig}
+        input_map[file] = {"cert": cert, "sig": sig, "bundle": bundle}
 
     if args.staging:
         logger.debug("verify: staging instances requested")
@@ -459,6 +531,12 @@ def _verify(args: argparse.Namespace) -> None:
         logger.debug(f"Using signature from: {inputs['sig']}")
         signature = inputs["sig"].read_bytes().rstrip()
 
+        entry: Optional[RekorEntry] = None
+        if inputs["bundle"].is_file():
+            logger.debug(f"Using offline Rekor bundle from: {inputs['bundle']}")
+            bundle = RekorBundle.parse_file(inputs["bundle"])
+            entry = bundle.to_entry()
+
         logger.debug(f"Verifying contents from: {file}")
 
         result = verifier.verify(
@@ -467,6 +545,7 @@ def _verify(args: argparse.Namespace) -> None:
             signature=signature,
             expected_cert_email=args.cert_email,
             expected_cert_oidc_issuer=args.cert_oidc_issuer,
+            offline_rekor_entry=entry,
         )
 
         if result:

diff --git a/sigstore/_internal/merkle.py b/sigstore/_internal/merkle.py
@@ -26,7 +26,7 @@
 import struct
 from typing import List, Tuple
 
-from sigstore._internal.rekor import RekorEntry, RekorInclusionProof
+from sigstore._internal.rekor import RekorEntry
 
 
 class InvalidInclusionProofError(Exception):
@@ -91,10 +91,11 @@ def _hash_leaf(leaf: bytes) -> bytes:
     return hashlib.sha256(data).digest()
 
 
-def verify_merkle_inclusion(
-    inclusion_proof: RekorInclusionProof, entry: RekorEntry
-) -> None:
+def verify_merkle_inclusion(entry: RekorEntry) -> None:
     """Verify the Merkle Inclusion Proof for a given Rekor entry"""
+    inclusion_proof = entry.inclusion_proof
+    if inclusion_proof is None:
+        raise InvalidInclusionProofError("Rekor entry has no inclusion proof")
 
     # Figure out which subset of hashes corresponds to the inner and border nodes.
     inner, border = _decomp_inclusion_proof(