-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore(NODE-6243): move Node release tooling to drivers-github-tools #45
Merged
baileympearson
merged 22 commits into
mongodb-labs:main
from
baileympearson:add-signing-env-action-for-node
Jun 27, 2024
Merged
Changes from all commits
Commits
Show all changes
22 commits
Select commit
Hold shift + click to select a range
344c9b5
add node actions:
baileympearson 0f18ab7
add node actions:
baileympearson d3eb67e
consolidate
baileympearson 6c43607
foo
baileympearson 6c1e028
npm
baileympearson e50eff7
add back in upload
baileympearson 5d14552
add setup action
baileympearson 5f53ade
add release template and release template generation
baileympearson 340513d
generation script
baileympearson 10a6b41
generation script
baileympearson aeef73c
signing fix
baileympearson c903292
add generic signing action
baileympearson 8d5ff35
shared signing action
baileympearson 508b0f1
make action shared
baileympearson 2f3357e
prettier changes
baileympearson c3c7739
address comments except for env
baileympearson b1e4e6f
revert env -> outputs change
baileympearson 8e44574
add debug information
baileympearson 4fa219d
use env
baileympearson b6962df
clean up unneeded debugging informatoin
baileympearson 5deae41
format
baileympearson 56ef850
adjust names
baileympearson File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,47 @@ | ||
import { readFileSync } from "node:fs"; | ||
import { join, dirname } from "node:path"; | ||
import { fileURLToPath } from "node:url"; | ||
|
||
const __dirname = dirname(fileURLToPath(import.meta.url)); | ||
|
||
const args = process.argv.slice(2); | ||
if (!(args.length === 3 || args.length === 4)) { | ||
console.error( | ||
`usage: generate_release.js <package> <branch> <npm tag> <optional silk asset group>`, | ||
); | ||
process.exit(1); | ||
} | ||
|
||
const [npmPackage, branch, tag, assetGroup] = args; | ||
|
||
const isNative = | ||
npmPackage === "kerberos" || npmPackage === "mongodb-client-encryption"; | ||
const template = readFileSync( | ||
join(__dirname, "./release_template.yml"), | ||
"utf-8", | ||
); | ||
|
||
const EVERGREEN_PROJECTS = { | ||
mongodb: "mongo-node-driver-next", | ||
bson: "js-bson", | ||
}; | ||
|
||
const generated = template | ||
.replaceAll("RELEASE_BRANCH", branch) | ||
.replaceAll("RELEASE_PACKAGE", npmPackage) | ||
.replaceAll("RELEASE_TAG", tag) | ||
.replaceAll("EVERGREEN_PROJECT", EVERGREEN_PROJECTS[npmPackage] ?? "") | ||
.replaceAll("IGNORE_INSTALL_SCRIPTS", isNative) | ||
.replaceAll("SILK_ASSET_GROUP", assetGroup ? `'${assetGroup}'` : "''"); | ||
|
||
const project = EVERGREEN_PROJECTS[npmPackage]; | ||
if (!project) { | ||
const final = generated | ||
.split("\n") | ||
.filter((line) => !line.includes("evergreen")) | ||
.join("\n"); | ||
process.stdout.write(final); | ||
process.exit(); | ||
} | ||
|
||
process.stdout.write(generated); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
name: Publish Release Asset to S3 | ||
description: "Publish Asset to S3" | ||
inputs: | ||
npm_package_name: | ||
description: the npm package name | ||
required: true | ||
|
||
runs: | ||
using: composite | ||
steps: | ||
- name: Get release version and release package file name | ||
id: get_version | ||
shell: bash | ||
run: | | ||
package_version=$(jq --raw-output '.version' package.json) | ||
echo "package_version=${package_version}" >> "$GITHUB_ENV" | ||
echo "package_file=${{ inputs.npm_package_name }}-${package_version}.tgz" >> "$GITHUB_ENV" | ||
echo "commit=$(git rev-parse HEAD)" >> $GITHUB_ENV |
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,105 @@ | ||
on: | ||
push: | ||
branches: [RELEASE_BRANCH] | ||
workflow_dispatch: {} | ||
|
||
permissions: | ||
contents: write | ||
pull-requests: write | ||
id-token: write | ||
|
||
name: release-RELEASE_TAG | ||
nbbeeken marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
jobs: | ||
release_please: | ||
runs-on: ubuntu-latest | ||
outputs: | ||
release_created: ${{ steps.release.outputs.release_created }} | ||
steps: | ||
- id: release | ||
uses: googleapis/release-please-action@v4 | ||
with: | ||
target-branch: RELEASE_BRANCH | ||
|
||
build: | ||
needs: [release_please] | ||
name: "Perform any build or bundling steps, as necessary." | ||
uses: ./.github/workflows/build.yml | ||
|
||
ssdlc: | ||
needs: [release_please, build] | ||
permissions: | ||
# required for all workflows | ||
security-events: write | ||
id-token: write | ||
contents: write | ||
environment: release | ||
runs-on: ubuntu-latest | ||
steps: | ||
- uses: actions/checkout@v4 | ||
|
||
- name: Install Node and dependencies | ||
uses: mongodb-labs/drivers-github-tools/node/setup@v2 | ||
with: | ||
ignore_install_scripts: IGNORE_INSTALL_SCRIPTS | ||
|
||
- name: Load version and package info | ||
uses: mongodb-labs/drivers-github-tools/node/get_version_info@v2 | ||
with: | ||
npm_package_name: RELEASE_PACKAGE | ||
|
||
- name: actions/compress_sign_and_upload | ||
uses: mongodb-labs/drivers-github-tools/node/sign_node_package@v2 | ||
with: | ||
aws_role_arn: ${{ secrets.AWS_ROLE_ARN }} | ||
aws_region_name: us-east-1 | ||
aws_secret_id: ${{ secrets.AWS_SECRET_ID }} | ||
npm_package_name: RELEASE_PACKAGE | ||
dry_run: ${{ needs.release_please.outputs.release_created == '' }} | ||
|
||
- name: Copy sbom file to release assets | ||
shell: bash | ||
if: ${{ SILK_ASSET_GROUP == '' }} | ||
run: cp sbom.json ${{ env.S3_ASSETS }}/sbom.json | ||
|
||
# only used for mongodb-client-encryption | ||
- name: Augment SBOM and copy to release assets | ||
if: ${{ SILK_ASSET_GROUP != '' }} | ||
uses: mongodb-labs/drivers-github-tools/sbom@v2 | ||
with: | ||
silk_asset_group: SILK_ASSET_GROUP | ||
sbom_file_name: sbom.json | ||
|
||
- name: Generate authorized pub report | ||
uses: mongodb-labs/drivers-github-tools/full-report@v2 | ||
with: | ||
release_version: ${{ env.package_version }} | ||
product_name: RELEASE_PACKAGE | ||
sarif_report_target_ref: RELEASE_BRANCH | ||
third_party_dependency_tool: n/a | ||
dist_filenames: artifacts/* | ||
token: ${{ github.token }} | ||
sbom_file_name: sbom.json | ||
evergreen_project: EVERGREEN_PROJECT | ||
evergreen_commit: ${{ env.commit }} | ||
|
||
- uses: mongodb-labs/drivers-github-tools/upload-s3-assets@v2 | ||
with: | ||
version: ${{ env.package_version }} | ||
product_name: RELEASE_PACKAGE | ||
dry_run: ${{ needs.release_please.outputs.release_created == '' }} | ||
|
||
publish: | ||
needs: [release_please, ssdlc, build] | ||
environment: release | ||
runs-on: ubuntu-latest | ||
steps: | ||
- uses: actions/checkout@v4 | ||
|
||
- name: Install Node and dependencies | ||
uses: mongodb-labs/drivers-github-tools/node/setup@v2 | ||
|
||
- run: npm publish --provenance --tag=RELEASE_TAG | ||
if: ${{ needs.release_please.outputs.release_created }} | ||
env: | ||
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
name: Setup | ||
description: "Installs node, driver dependencies, and builds source" | ||
inputs: | ||
ignore_install_scripts: | ||
description: Should we ignore postinstall scripts? | ||
default: "false" | ||
|
||
runs: | ||
using: composite | ||
steps: | ||
- uses: actions/setup-node@v4 | ||
with: | ||
node-version: "lts/*" | ||
cache: "npm" | ||
registry-url: "https://registry.npmjs.org" | ||
- run: npm install -g npm@latest | ||
shell: bash | ||
- run: npm clean-install | ||
nbbeeken marked this conversation as resolved.
Show resolved
Hide resolved
|
||
if: ${{ inputs.ignore_install_scripts == 'false' }} | ||
shell: bash | ||
- run: npm clean-install --ignore-scripts | ||
if: ${{ inputs.ignore_install_scripts == 'true' }} | ||
shell: bash |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,90 @@ | ||
name: Compress, Sign and Upload to GH Release | ||
description: "Compresses package and signs with garasign and uploads to GH release" | ||
|
||
inputs: | ||
aws_role_arn: | ||
description: "AWS role input for drivers-github-tools/gpg-sign@v2" | ||
required: true | ||
aws_region_name: | ||
description: "AWS region name input for drivers-github-tools/gpg-sign@v2" | ||
required: true | ||
aws_secret_id: | ||
description: "AWS secret id input for drivers-github-tools/gpg-sign@v2" | ||
required: true | ||
npm_package_name: | ||
description: "The name for the npm package this repository represents" | ||
required: true | ||
dry_run: | ||
description: "Should we upload files to the release?" | ||
required: false | ||
default: "false" | ||
artifact_directory: | ||
description: The directory in which to output signatures. | ||
default: artifacts | ||
sign_native: | ||
description: Download and sign native packages | ||
default: "false" | ||
|
||
runs: | ||
using: composite | ||
steps: | ||
- uses: actions/download-artifact@v4 | ||
if: ${{ inputs.sign_native == 'true' }} | ||
|
||
- run: npm pack | ||
shell: bash | ||
|
||
- name: Make signatures directory | ||
shell: bash | ||
run: | | ||
mkdir ${{ inputs.artifact_directory }} | ||
|
||
- name: Load version and package info | ||
uses: mongodb-labs/drivers-github-tools/node/get_version_info@v2 | ||
with: | ||
npm_package_name: ${{ inputs.npm_package_name }} | ||
|
||
- name: Set up drivers-github-tools | ||
uses: mongodb-labs/drivers-github-tools/setup@v2 | ||
with: | ||
aws_region_name: ${{ inputs.aws_region_name }} | ||
aws_role_arn: ${{ inputs.aws_role_arn }} | ||
aws_secret_id: ${{ inputs.aws_secret_id }} | ||
|
||
- name: Determine what files to sign (native packages, works with glob patterns of build artifacts) | ||
if: ${{ inputs.sign_native == 'true' }} | ||
shell: bash | ||
run: | | ||
FILENAMES="build-*/*.tar.gz" | ||
if [[ $FILENAMES =~ '*' ]]; then | ||
FILENAMES=$(ls $FILENAMES | sed -e ':a' -e 'N' -e '$!ba' -e 's/\n/ /g') | ||
fi | ||
FILENAMES="$FILENAMES ${{ env.package_file }}" | ||
echo "FILES_TO_SIGN=${FILENAMES}" >> "$GITHUB_ENV" | ||
|
||
- name: Determine what files to sign (non-native packages, with only the release tarball) | ||
if: ${{ inputs.sign_native != 'true' }} | ||
shell: bash | ||
run: | | ||
FILENAMES="${{ env.package_file }}" | ||
echo "FILES_TO_SIGN=${FILENAMES}" >> "$GITHUB_ENV" | ||
|
||
- name: Create detached signature | ||
uses: mongodb-labs/drivers-github-tools/gpg-sign@v2 | ||
with: | ||
filenames: ${{ env.FILES_TO_SIGN }} | ||
env: | ||
RELEASE_ASSETS: ${{ inputs.artifact_directory }} | ||
|
||
- name: Copy the tarballs to the artifacts directory | ||
shell: bash | ||
run: | | ||
for filename in ${{ env.FILES_TO_SIGN }}; do cp ${filename} artifacts/; done | ||
ls -la artifacts/ | ||
|
||
- name: "Upload release artifacts" | ||
if: ${{ inputs.dry_run == false }} | ||
run: gh release upload v${{ env.package_version }} artifacts/*.* | ||
shell: bash | ||
env: | ||
GH_TOKEN: ${{ github.token }} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why did this change to ENV vs OUTPUT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ENV is easier to work with because anything can be accessed with
env.<NAME>
and doesn't need to worry about step ids.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It may be easier to use but it removes the explicit linkages of dependence between steps, the need to reference ids is a feature not a burden
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note that as per the Workflow syntax for GitHub Actions, steps don't have outputs. The syntax
steps.<step-id>.outputs.<output-id>
is used to access outputs from an action called in that particular step. The$GITHUB_OUTPUT
env variable is shared between all steps in a job, so it's no different from$GITHUB_ENV
, with the subtle difference that$GITHUB_ENV
is not made available as a job output.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry a bit confused here, each step can have outputs, the
steps
context is how you access them. Here's an example of how we use them in the driverWhen you've called away to an action, it can return specified values by declaring its outputs: https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#outputs-for-composite-actions
Whether you're calling away to an external action or running an inline script step using outputs is an alternative to using
env
. The benefit of outputs is the requirement to declaratively connect your ins and outs.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Welp, of course it'd be in the step context, not in the workflow syntax. I stand corrected - I assumed the outputs were only available when calling to a separate action in a step.