Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update cloud docs for RBE and Read Only #1322

Merged
merged 1 commit into from
Sep 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ challenging. Feel free to copy-paste it into your own projects.
See [`docs/package.json`](https://github.com/TraceMachina/nativelink/blob/main/docs/package.json)
for build scripts.

This project requires `bun`. The nix flake ships a compatible version.
This project requires `bun` and `deno`. The nix flake ships compatible versions.

```bash
# Install dependencies with bun.
Expand Down
26 changes: 16 additions & 10 deletions docs/src/content/docs/nativelink-cloud/api-key.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -45,21 +45,27 @@ GitHub Repository Secrets is the recommended way
to store your NativeLink Cloud API keys
for use with GitHub Actions.

[GitHub Repository Secrets Tutorial](https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions#creating-secrets-for-a-repository)

They can be access via the secrets context i.e.
The [GitHub repository secrets tutorial](https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions#creating-secrets-for-a-repository) will show you how to setup a repository secret. You can then access those variables from the secrets context in your GitHub actions:
```bash
${{ secrets.YourSecretKeyName }}
```

:::note
If your repository uses forks for pull requests (PRs), note that the secrets
context isn't accessible from PRs originating from forks. To address this,
we introduced Read-Only keys.
:::
#### Repositories that use forks
If your repository is open source and uses forks for pull requests (PRs), the secrets and vars
context aren't accessible in the actions for PRs originating from the forks. To address this,
we recommend using Read-Only keys.

These keys can be generated in the `Settings > API Keys & Certs` sections of the cloud dashboard.

Read-Only keys can be used for PRs and stored in the vars context. For
merge-to-main actions, use a Read/Write key stored in the secrets context.
Read-Only keys can be used for PRs and be hard-coded into your GitHub actions. For
merge-to-main actions, use a Read/Write key stored in the secrets context. An example of this below:
```bash
bazel test \
--remote_cache=grpcs://cas-account-id.build-faster.nativelink.net \
--remote_header=x-nativelink-api-key=${{ secrets.NATIVELINK_COM_API_HEADER || 'HARD_CODED_READ_ONLY_KEY_HERE' }} \
${{ github.ref == 'refs/heads/main' && ' ' || '--nogenerate_json_trace_profile --remote_upload_local_results=false' }} \
//...
```

## Read Only keys

Expand Down
67 changes: 67 additions & 0 deletions docs/src/content/docs/nativelink-cloud/rbe.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
---
title: "Remote Build Execution"
description: "Utilize Bazel's RBE protocol for remote builds in the NativeLink Cloud"
pagefind: true
---
:::note
NativeLink Cloud RBE is currently in limited release. Please request access from the
Remote Execution page in the NativeLink Cloud to enable RBE.
:::

This guide shows how to configure remote build execution (RBE) for your
[Bazel](https://bazel.build/) projects with the
[NativeLink Cloud](https://app.nativelink.com). Before using this guide
make sure you have followed our [Bazel Quickstart](/nativelink-cloud/bazel).

### Basic Configuration
To enable RBE all you need to do is add the below flag to your Bazel builds:
```bash
--remote_executor=grpcs://scheduler-YOUR_ACCOUNT_HERE.build-faster.nativelink.net:443
```

This will run your builds on a Ubuntu 22.04 image *without any* dependencies installed.
For most users we don't expect this to work out of the box as your project most
likely depends on installations like GCC/Java/etc. To remedy that, continue with the
instructions below to pass in your own images.

### Custom Images
To support most RBE builds you will most likely need to pass in your own image with the
correct toolchains installed to support your build. To implement
that we've the below flag configuration:
```bash
--remote_default_exec_properties="container-image=docker://public.ecr.aws/ubuntu/ubuntu:22.04_stable"
```
:::note
The above uses a public AWS ECR image for example
:::

#### Private Images
If your images are in your own private image repository, you can pass your repository
credentials to allow us to pull your RBE images.

import { Tabs, TabItem } from '@astrojs/starlight/components';

<Tabs syncKey="platfomr">
<TabItem label="AWS">
```bash
--remote_default_exec_properties="container-image=docker://123456789100.dkr.ecr.us-east-2.amazonaws.com/rbe-images:tag"
--remote_exec_header=x-nativelink-rbe.container-registry-username=AWS
--remote_exec_header=x-nativelink-rbe.container-registry-password="$(aws ecr get-login-password --region YOUR_REGION)"
```
</TabItem>
<TabItem label="Google Cloud">
```bash
--remote_default_exec_properties="container-image=docker://gcr.io/rbe-images/image"
--remote_exec_header=x-nativelink-rbe.container-registry-username=_dcgcloud_token
--remote_exec_header=x-nativelink-rbe.container-registry-password="$(gcloud auth print-access-token)"
```
</TabItem>
</Tabs>

### Hermetic Bazel Builds
An alternative option to passing in your own custom image is using a fully hermetic
Bazel build. This will allow you to use our default Ubuntu 22.04 image and your Bazel
commands will install all needed dependencies.

You can see a sample of that in the WORKSPACE file of our **Hermetic CC** example
repository [here](https://github.com/TraceMachina/hermetic_cc_toolchain_rbe_example/blob/main/WORKSPACE).
4 changes: 4 additions & 0 deletions docs/starlight.conf.ts
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,10 @@ export const starlightConfig = {
label: "Pants",
link: `${docsRoot}/nativelink-cloud/pants`,
},
{
label: "Remote Build Execution",
link: "/nativelink-cloud/rbe/",
},
{
label: "API Keys in CI",
link: `${docsRoot}/nativelink-cloud/api-key`,
Expand Down
Loading