Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(NODE-6243): move Node release tooling to drivers-github-tools #45

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion gpg-sign/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ runs:
id: filenames
run: |
set -eux
export FILENAMES=${{inputs.filenames}}
FILENAMES="${{inputs.filenames}}"
if [[ $FILENAMES =~ '*' ]]; then
FILENAMES=$(ls $FILENAMES | sed -e ':a' -e 'N' -e '$!ba' -e 's/\n/ /g')
fi
Expand Down
47 changes: 47 additions & 0 deletions node/generate_release.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
import { readFileSync } from "node:fs";
import { join, dirname } from "node:path";
import { fileURLToPath } from "node:url";

const __dirname = dirname(fileURLToPath(import.meta.url));

const args = process.argv.slice(2);
if (!(args.length === 3 || args.length === 4)) {
console.error(
`usage: generate_release.js <package> <branch> <npm tag> <optional silk asset group>`,
);
process.exit(1);
}

const [npmPackage, branch, tag, assetGroup] = args;

const isNative =
npmPackage === "kerberos" || npmPackage === "mongodb-client-encryption";
const template = readFileSync(
join(__dirname, "./release_template.yml"),
"utf-8",
);

const EVERGREEN_PROJECTS = {
mongodb: "mongo-node-driver-next",
bson: "js-bson",
};

const generated = template
.replaceAll("RELEASE_BRANCH", branch)
.replaceAll("RELEASE_PACKAGE", npmPackage)
.replaceAll("RELEASE_TAG", tag)
.replaceAll("EVERGREEN_PROJECT", EVERGREEN_PROJECTS[npmPackage] ?? "")
.replaceAll("IGNORE_INSTALL_SCRIPTS", isNative)
.replaceAll("SILK_ASSET_GROUP", assetGroup ? `'${assetGroup}'` : "''");

const project = EVERGREEN_PROJECTS[npmPackage];
if (!project) {
const final = generated
.split("\n")
.filter((line) => !line.includes("evergreen"))
.join("\n");
process.stdout.write(final);
process.exit();
}

process.stdout.write(generated);
18 changes: 18 additions & 0 deletions node/get_version_info/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
name: Publish Release Asset to S3
description: "Publish Asset to S3"
inputs:
npm_package_name:
description: the npm package name
required: true

runs:
using: composite
steps:
- name: Get release version and release package file name
id: get_version
shell: bash
run: |
package_version=$(jq --raw-output '.version' package.json)
echo "package_version=${package_version}" >> "$GITHUB_ENV"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did this change to ENV vs OUTPUT?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ENV is easier to work with because anything can be accessed with env.<NAME> and doesn't need to worry about step ids.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may be easier to use but it removes the explicit linkages of dependence between steps, the need to reference ids is a feature not a burden

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that as per the Workflow syntax for GitHub Actions, steps don't have outputs. The syntax steps.<step-id>.outputs.<output-id> is used to access outputs from an action called in that particular step. The $GITHUB_OUTPUT env variable is shared between all steps in a job, so it's no different from $GITHUB_ENV, with the subtle difference that $GITHUB_ENV is not made available as a job output.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

steps don't have outputs.

Sorry a bit confused here, each step can have outputs, the steps context is how you access them. Here's an example of how we use them in the driver

used to access outputs from an action called in that particular step

When you've called away to an action, it can return specified values by declaring its outputs: https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#outputs-for-composite-actions

Whether you're calling away to an external action or running an inline script step using outputs is an alternative to using env. The benefit of outputs is the requirement to declaratively connect your ins and outs.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[...] the steps context is how you access them

Welp, of course it'd be in the step context, not in the workflow syntax. I stand corrected - I assumed the outputs were only available when calling to a separate action in a step.

echo "package_file=${{ inputs.npm_package_name }}-${package_version}.tgz" >> "$GITHUB_ENV"
echo "commit=$(git rev-parse HEAD)" >> $GITHUB_ENV
39 changes: 0 additions & 39 deletions node/publish_asset_to_s3/action.yml

This file was deleted.

105 changes: 105 additions & 0 deletions node/release_template.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
on:
push:
branches: [RELEASE_BRANCH]
workflow_dispatch: {}

permissions:
contents: write
pull-requests: write
id-token: write

name: release-RELEASE_TAG
nbbeeken marked this conversation as resolved.
Show resolved Hide resolved

jobs:
release_please:
runs-on: ubuntu-latest
outputs:
release_created: ${{ steps.release.outputs.release_created }}
steps:
- id: release
uses: googleapis/release-please-action@v4
with:
target-branch: RELEASE_BRANCH

build:
needs: [release_please]
name: "Perform any build or bundling steps, as necessary."
uses: ./.github/workflows/build.yml

ssdlc:
needs: [release_please, build]
permissions:
# required for all workflows
security-events: write
id-token: write
contents: write
environment: release
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Install Node and dependencies
uses: mongodb-labs/drivers-github-tools/node/setup@v2
with:
ignore_install_scripts: IGNORE_INSTALL_SCRIPTS

- name: Load version and package info
uses: mongodb-labs/drivers-github-tools/node/get_version_info@v2
with:
npm_package_name: RELEASE_PACKAGE

- name: actions/compress_sign_and_upload
uses: mongodb-labs/drivers-github-tools/node/sign_node_package@v2
with:
aws_role_arn: ${{ secrets.AWS_ROLE_ARN }}
aws_region_name: us-east-1
aws_secret_id: ${{ secrets.AWS_SECRET_ID }}
npm_package_name: RELEASE_PACKAGE
dry_run: ${{ needs.release_please.outputs.release_created == '' }}

- name: Copy sbom file to release assets
shell: bash
if: ${{ SILK_ASSET_GROUP == '' }}
run: cp sbom.json ${{ env.S3_ASSETS }}/sbom.json

# only used for mongodb-client-encryption
- name: Augment SBOM and copy to release assets
if: ${{ SILK_ASSET_GROUP != '' }}
uses: mongodb-labs/drivers-github-tools/sbom@v2
with:
silk_asset_group: SILK_ASSET_GROUP
sbom_file_name: sbom.json

- name: Generate authorized pub report
uses: mongodb-labs/drivers-github-tools/full-report@v2
with:
release_version: ${{ env.package_version }}
product_name: RELEASE_PACKAGE
sarif_report_target_ref: RELEASE_BRANCH
third_party_dependency_tool: n/a
dist_filenames: artifacts/*
token: ${{ github.token }}
sbom_file_name: sbom.json
evergreen_project: EVERGREEN_PROJECT
evergreen_commit: ${{ env.commit }}

- uses: mongodb-labs/drivers-github-tools/upload-s3-assets@v2
with:
version: ${{ env.package_version }}
product_name: RELEASE_PACKAGE
dry_run: ${{ needs.release_please.outputs.release_created == '' }}

publish:
needs: [release_please, ssdlc, build]
environment: release
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Install Node and dependencies
uses: mongodb-labs/drivers-github-tools/node/setup@v2

- run: npm publish --provenance --tag=RELEASE_TAG
if: ${{ needs.release_please.outputs.release_created }}
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
23 changes: 23 additions & 0 deletions node/setup/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
name: Setup
description: "Installs node, driver dependencies, and builds source"
inputs:
ignore_install_scripts:
description: Should we ignore postinstall scripts?
default: "false"

runs:
using: composite
steps:
- uses: actions/setup-node@v4
with:
node-version: "lts/*"
cache: "npm"
registry-url: "https://registry.npmjs.org"
- run: npm install -g npm@latest
shell: bash
- run: npm clean-install
nbbeeken marked this conversation as resolved.
Show resolved Hide resolved
if: ${{ inputs.ignore_install_scripts == 'false' }}
shell: bash
- run: npm clean-install --ignore-scripts
if: ${{ inputs.ignore_install_scripts == 'true' }}
shell: bash
90 changes: 90 additions & 0 deletions node/sign_node_package/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
name: Compress, Sign and Upload to GH Release
description: "Compresses package and signs with garasign and uploads to GH release"

inputs:
aws_role_arn:
description: "AWS role input for drivers-github-tools/gpg-sign@v2"
required: true
aws_region_name:
description: "AWS region name input for drivers-github-tools/gpg-sign@v2"
required: true
aws_secret_id:
description: "AWS secret id input for drivers-github-tools/gpg-sign@v2"
required: true
npm_package_name:
description: "The name for the npm package this repository represents"
required: true
dry_run:
description: "Should we upload files to the release?"
required: false
default: "false"
artifact_directory:
description: The directory in which to output signatures.
default: artifacts
sign_native:
description: Download and sign native packages
default: "false"

runs:
using: composite
steps:
- uses: actions/download-artifact@v4
if: ${{ inputs.sign_native == 'true' }}

- run: npm pack
shell: bash

- name: Make signatures directory
shell: bash
run: |
mkdir ${{ inputs.artifact_directory }}

- name: Load version and package info
uses: mongodb-labs/drivers-github-tools/node/get_version_info@v2
with:
npm_package_name: ${{ inputs.npm_package_name }}

- name: Set up drivers-github-tools
uses: mongodb-labs/drivers-github-tools/setup@v2
with:
aws_region_name: ${{ inputs.aws_region_name }}
aws_role_arn: ${{ inputs.aws_role_arn }}
aws_secret_id: ${{ inputs.aws_secret_id }}

- name: Determine what files to sign (native packages, works with glob patterns of build artifacts)
if: ${{ inputs.sign_native == 'true' }}
shell: bash
run: |
FILENAMES="build-*/*.tar.gz"
if [[ $FILENAMES =~ '*' ]]; then
FILENAMES=$(ls $FILENAMES | sed -e ':a' -e 'N' -e '$!ba' -e 's/\n/ /g')
fi
FILENAMES="$FILENAMES ${{ env.package_file }}"
echo "FILES_TO_SIGN=${FILENAMES}" >> "$GITHUB_ENV"

- name: Determine what files to sign (non-native packages, with only the release tarball)
if: ${{ inputs.sign_native != 'true' }}
shell: bash
run: |
FILENAMES="${{ env.package_file }}"
echo "FILES_TO_SIGN=${FILENAMES}" >> "$GITHUB_ENV"

- name: Create detached signature
uses: mongodb-labs/drivers-github-tools/gpg-sign@v2
with:
filenames: ${{ env.FILES_TO_SIGN }}
env:
RELEASE_ASSETS: ${{ inputs.artifact_directory }}

- name: Copy the tarballs to the artifacts directory
shell: bash
run: |
for filename in ${{ env.FILES_TO_SIGN }}; do cp ${filename} artifacts/; done
ls -la artifacts/

- name: "Upload release artifacts"
if: ${{ inputs.dry_run == false }}
run: gh release upload v${{ env.package_version }} artifacts/*.*
shell: bash
env:
GH_TOKEN: ${{ github.token }}
Loading