Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store firmware binaries in CAS #156

Merged
merged 8 commits into from
Nov 1, 2023
Merged

Store firmware binaries in CAS #156

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
1896060
Update Makefile
AlCutter Oct 31, 2023
bc922e2
Update cloudbuild configs
AlCutter Oct 31, 2023
e21a85b
bump armored-witness-common to eefcf9dd7f27a07993d1cf612ab3e798d08d316e
AlCutter Oct 31, 2023
c53015a
mod tidy
AlCutter Oct 31, 2023
88fc5d2
use subdir for CAS
AlCutter Nov 1, 2023
2db3165
Don't store manifest in bucket
AlCutter Nov 1, 2023
c643915
Remove unneeded substitutions
AlCutter Nov 1, 2023
1ebb679
update bin url for dev
AlCutter Nov 1, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ BUILD = ${BUILD_USER}@${BUILD_HOST} on ${BUILD_DATE}
REV = $(shell git rev-parse --short HEAD 2> /dev/null)
DEV_LOG_ORIGIN ?= "DEV.armoredwitness.transparency.dev/${USER}"
GIT_SEMVER_TAG ?= $(shell (git describe --tags --exact-match --match 'v*.*.*' 2>/dev/null || git describe --match 'v*.*.*' --tags 2>/dev/null || git describe --tags 2>/dev/null || echo -n v0.0.${BUILD_EPOCH}+`git rev-parse HEAD`) | tail -c +2 )
FT_BIN_URL ?= "http://$(shell hostname --fqdn):9944/"
FT_BIN_URL ?= "http://$(shell hostname --fqdn):9944/artefacts/"
FT_LOG_URL ?= "http://$(shell hostname --fqdn):9944/log/"
FT_LOG_ORIGIN ?= $(DEV_LOG_ORIGIN)

Expand Down Expand Up @@ -87,7 +87,8 @@ log_initialise:

## log_applet adds the trusted_applet_manifest file created during the build to the dev FT log.
log_applet: LOG_STORAGE_DIR=$(DEV_LOG_DIR)/log
log_applet: LOG_ARTEFACT_DIR=$(DEV_LOG_DIR)/trusted-applet/$(GIT_SEMVER_TAG)
log_applet: LOG_ARTEFACT_DIR=$(DEV_LOG_DIR)/artefacts
log_applet: ARTEFACT_HASH=$(shell sha256sum ${CURDIR}/bin/trusted_applet.elf | cut -f1 -d" ")
log_applet:
@if [ "${LOG_PRIVATE_KEY}" == "" -o "${LOG_PUBLIC_KEY}" == "" ]; then \
@echo "You need to set LOG_PRIVATE_KEY and LOG_PUBLIC_KEY variables"; \
Expand All @@ -112,7 +113,7 @@ log_applet:
--private_key=${LOG_PRIVATE_KEY} \
--public_key=${LOG_PUBLIC_KEY}
@mkdir -p ${LOG_ARTEFACT_DIR}
cp ${CURDIR}/bin/trusted_applet.* ${LOG_ARTEFACT_DIR}
cp ${CURDIR}/bin/trusted_applet.elf ${LOG_ARTEFACT_DIR}/${ARTEFACT_HASH}

#### ARM targets ####

Expand Down
2 changes: 1 addition & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ require (
github.com/machinebox/progress v0.2.0
github.com/prometheus/client_golang v1.17.0
github.com/transparency-dev/armored-witness-boot v0.0.0-20230904140406-e2e16c7665b7
github.com/transparency-dev/armored-witness-common v0.0.0-20231027110430-3802c9e3e15d
github.com/transparency-dev/armored-witness-common v0.0.0-20231031160117-eefcf9dd7f27
github.com/transparency-dev/armored-witness-os v0.0.0-20230904142303-8cff7e12c215
github.com/transparency-dev/formats v0.0.0-20230920083814-0f75b1d4e813
github.com/transparency-dev/merkle v0.0.2
Expand Down
4 changes: 2 additions & 2 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,8 +72,8 @@ github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcU
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/transparency-dev/armored-witness-boot v0.0.0-20230904140406-e2e16c7665b7 h1:3xrmiN4hwWi3nxvDo9asUWrNCjaPYBhF+rHpW97Fde0=
github.com/transparency-dev/armored-witness-boot v0.0.0-20230904140406-e2e16c7665b7/go.mod h1:GTj2zM9nwFe7G7gaXzIbkKJ/PkZfvVq4TdNiA6CBsdo=
github.com/transparency-dev/armored-witness-common v0.0.0-20231027110430-3802c9e3e15d h1:76yBTOSuqGXpKe7qjF1Yjzf4MOhXYryeVObzUkpKHf0=
github.com/transparency-dev/armored-witness-common v0.0.0-20231027110430-3802c9e3e15d/go.mod h1:6M39UQVYzzFdzXGrJFjRC+G9D5f0icxOVkcX/yaWZss=
github.com/transparency-dev/armored-witness-common v0.0.0-20231031160117-eefcf9dd7f27 h1:p8mmHwCvTYbuB52ph9knjwWkQmGNZ+3BZJgsw9xIQq0=
github.com/transparency-dev/armored-witness-common v0.0.0-20231031160117-eefcf9dd7f27/go.mod h1:6M39UQVYzzFdzXGrJFjRC+G9D5f0icxOVkcX/yaWZss=
github.com/transparency-dev/armored-witness-os v0.0.0-20230904142303-8cff7e12c215 h1:xY5bolI/XmV9sBStzn8rSXV8E4foG0pGZAaYUKdrGHc=
github.com/transparency-dev/armored-witness-os v0.0.0-20230904142303-8cff7e12c215/go.mod h1:WfH1eII946tTqjniWgozuWbJthgDEuSaeGE5jta+3Ew=
github.com/transparency-dev/formats v0.0.0-20230920083814-0f75b1d4e813 h1:PHklaeYyhPsbhWt+MnKpBvJrsJGkPEaU1JutMj4wNqM=
Expand Down
4 changes: 3 additions & 1 deletion release/cloudbuild.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,13 @@ steps:
- output
# Copy the artifacts from the Cloud Build VM to GCS.
- name: gcr.io/cloud-builders/gcloud
entrypoint: bash
args:
- gcloud
Comment on lines +39 to +41
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like @jiggoha to take a look at this entrypoint swapping business.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the reason Al has added that here is so that the sha256sum and other bash commands work correctly. The environment variables here get populated correctly not because of bash environment variables, but because of Cloud Build substitution vars. So without the entrypoint, we can't do any logic here. Is that right, Al?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, that's right

- storage
- cp
- output/trusted_applet.elf
- gs://${_FIRMWARE_BUCKET}/${_FIRMWARE_COMPONENT}/${TAG_NAME}/trusted_applet.elf
- gs://${_FIRMWARE_BUCKET}/$(sha256sum output/trusted_applet.elf | cut -f1 -d" ")
### Construct log entry / Claimant Model statement.
- name: golang
args:
Expand Down
16 changes: 3 additions & 13 deletions release/cloudbuild_ci.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,13 @@ steps:
- output
# Copy the artifacts from the Cloud Build VM to GCS.
- name: gcr.io/cloud-builders/gcloud
entrypoint: bash
args:
- gcloud
- storage
- cp
- output/trusted_applet.elf
- gs://${_FIRMWARE_BUCKET}/${_FIRMWARE_COMPONENT}/${_TEST_TAG_NAME}/trusted_applet.elf
- gs://${_FIRMWARE_BUCKET}/$(sha256sum output/trusted_applet.elf | cut -f1 -d" ")
### Construct log entry / Claimant Model statement.
- name: golang
args:
Expand Down Expand Up @@ -81,13 +83,6 @@ steps:
args:
- cat
- output/trusted_applet_manifest
### Copy the signed manifest to the public artifacts bucket.
- name: gcr.io/cloud-builders/gcloud
args:
- storage
- cp
- output/trusted_applet_manifest
- gs://${_FIRMWARE_BUCKET}/${_FIRMWARE_COMPONENT}/${_TEST_TAG_NAME}/trusted_applet_manifest
### Write the firmware release to the transparency log.
# Copy the log entry to the sequence bucket, preparing to write to log.
- name: gcr.io/cloud-builders/gcloud
Expand Down Expand Up @@ -118,11 +113,6 @@ substitutions:
_FIRMWARE_COMPONENT: trusted-applet
_TAMAGO_VERSION: '1.20.6'
_TEST_TAG_NAME: '0.1.2'
# Signing-related.
_REGION: global
_KMS_APPLET_KEY: trusted-applet-ci
_KMS_APPLET_KEY_VERSION: '1'
_KMS_KEYRING: firmware-release-ci
# Log-related.
_ENTRIES_DIR: firmware-log-sequence
_ORIGIN: transparency.dev/armored-witness/firmware_transparency/ci/0
Expand Down