From 326cdfaf344c19cdd21a93b789364539d4cb5685 Mon Sep 17 00:00:00 2001 From: Brian Clark Date: Fri, 30 Aug 2024 12:18:43 -0700 Subject: [PATCH] Update cloud docs for RBE and Read Only --- docs/README.md | 2 +- .../content/docs/nativelink-cloud/api-key.mdx | 26 ++++++---- .../src/content/docs/nativelink-cloud/rbe.mdx | 50 +++++++++++++++++++ docs/starlight.conf.ts | 4 ++ 4 files changed, 71 insertions(+), 11 deletions(-) create mode 100644 docs/src/content/docs/nativelink-cloud/rbe.mdx diff --git a/docs/README.md b/docs/README.md index faace677fa..db6f233d66 100644 --- a/docs/README.md +++ b/docs/README.md @@ -23,7 +23,7 @@ challenging. Feel free to copy-paste it into your own projects. See [`docs/package.json`](https://github.com/TraceMachina/nativelink/blob/main/docs/package.json) for build scripts. -This project requires `bun`. The nix flake ships a compatible version. +This project requires `bun` and `deno`. The nix flake ships a compatible version of `bun`. ```bash # Install dependencies with bun. diff --git a/docs/src/content/docs/nativelink-cloud/api-key.mdx b/docs/src/content/docs/nativelink-cloud/api-key.mdx index 724a9b51fb..510eaa3c1a 100644 --- a/docs/src/content/docs/nativelink-cloud/api-key.mdx +++ b/docs/src/content/docs/nativelink-cloud/api-key.mdx @@ -45,21 +45,27 @@ GitHub Repository Secrets is the recommended way to store your NativeLink Cloud API keys for use with GitHub Actions. -[GitHub Repository Secrets Tutorial](https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions#creating-secrets-for-a-repository) - -They can be access via the secrets context i.e. +The [GitHub repository secrets tutorial](https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions#creating-secrets-for-a-repository) will show you how to setup a repository secret. You can then access those variables from the secrets context in your GitHub actions: ```bash ${{ secrets.YourSecretKeyName }} ``` -:::note -If your repository uses forks for pull requests (PRs), note that the secrets -context isn't accessible from PRs originating from forks. To address this, -we introduced Read-Only keys. -::: +#### Repositories that use forks +If your repository is open source and uses forks for pull requests (PRs), the secrets and vars +context aren't accessible in the actions for PRs originating from the forks. To address this, +we recommend using Read-Only keys. + +These keys can be generated in the `Settings > API Keys & Certs` sections of the cloud dashboard. -Read-Only keys can be used for PRs and stored in the vars context. For -merge-to-main actions, use a Read/Write key stored in the secrets context. +Read-Only keys can be used for PRs and be hard-coded into your GitHub actions. For +merge-to-main actions, use a Read/Write key stored in the secrets context. An example of this below: +```bash +bazel test \ + --remote_cache=grpcs://cas-account-id.build-faster.nativelink.net \ + --remote_header=x-nativelink-api-key=${{ secrets.NATIVELINK_COM_API_HEADER || 'HARD_CODED_READ_ONLY_KEY_HERE' }} \ + ${{ github.ref == 'refs/heads/main' && ' ' || '--nogenerate_json_trace_profile --remote_upload_local_results=false' }} \ + //... +``` ## Read Only keys diff --git a/docs/src/content/docs/nativelink-cloud/rbe.mdx b/docs/src/content/docs/nativelink-cloud/rbe.mdx new file mode 100644 index 0000000000..9e5267bb50 --- /dev/null +++ b/docs/src/content/docs/nativelink-cloud/rbe.mdx @@ -0,0 +1,50 @@ +--- +title: "Remote Build Execution" +description: "Utilize Bazel's RBE protocol for remote builds in the NativeLink Cloud" +pagefind: true +--- +:::note +NativeLink Cloud RBE is currently in limited release. Please request access from the Remote Execution page in the NativeLink Cloud to enable RBE. +::: + +This guide shows how to configure remote build execution (RBE) for your [Bazel](https://bazel.build/) projects with the +[NativeLink Cloud](https://app.nativelink.com). Before using this guide make sure you have followed our [Bazel Quickstart](/nativelink-cloud/bazel). + +### Basic Configuration +To enable RBE all you need to do is add the below flag to your Bazel builds: +```bash +--remote_executor=grpcs://scheduler-YOUR_ACCOUNT_HERE.build-faster.nativelink.net:443 +``` + +This will run your builds on a Ubuntu 22.04 image *without any* dependencies installed. For most customers we don't expect this to work out of the box as your project most likely depends on installations like GCC/Java/etc. To remedy that, continue with the instructions below to pass in your own images. + +### Custom Images +To support most RBE builds you will most likely need to pass in your own image with the correct toolchains installed to support your build. To implement that we've the below flag configuration: +```bash +--remote_default_exec_properties="container-image=docker://public.ecr.aws/ubuntu/ubuntu:22.04_stable" +``` +:::note +The above uses a public AWS ECR image for example +::: + +#### Private Images +If your images are in your own private image repository, you can pass your repository credentials to allow us to pull your RBE images. + +Here's an example with Amazon ECR (Elastic Container Registry) and the AWS CLI: +```bash +--remote_default_exec_properties="container-image=docker://123456789100.dkr.ecr.us-east-2.amazonaws.com/rbe-images:tag" +--remote_exec_header=x-nativelink-rbe.container-registry-username=AWS +--remote_exec_header=x-nativelink-rbe.container-registry-password="$(aws ecr get-login-password --region YOUR_REGION)" +``` + +And another example with Google Cloud GCR (Google Container Registry) and the Google Cloud CLI: +```bash +--remote_default_exec_properties="container-image=docker://gcr.io/rbe-images/image" +--remote_exec_header=x-nativelink-rbe.container-registry-username=_dcgcloud_token +--remote_exec_header=x-nativelink-rbe.container-registry-password="$(gcloud auth print-access-token)" +``` + +### Hermetic Bazel Builds +An alternative option to passing in your own custom image is using a fully hermetic Bazel build. This will allow you to use our default Ubuntu 22.04 image and your Bazel commands will install all needed dependencies. + +You can see a sample of that in the WORKSPACE file of our **Hermetic CC** example repository [here](https://github.com/TraceMachina/hermetic_cc_toolchain_rbe_example/blob/main/WORKSPACE). diff --git a/docs/starlight.conf.ts b/docs/starlight.conf.ts index 08b27202ef..dcda3f7a12 100644 --- a/docs/starlight.conf.ts +++ b/docs/starlight.conf.ts @@ -69,6 +69,10 @@ export const starlightConfig = { label: "Pants", link: `${docsRoot}/nativelink-cloud/pants`, }, + { + label: "Remote Build Execution", + link: "/nativelink-cloud/rbe/", + }, { label: "API Keys in CI", link: `${docsRoot}/nativelink-cloud/api-key`,