Vault Plugin Secrets GitHub

About
- Why?
- What?
- How?
Installation
API
- Token
- Permission sets
- Config
- Metrics
- Info
Development
- Tests
Security

About

Are you using HashiCorp Vault and GitHub in your organisation? Do you want ephemeral, finely-scoped GitHub tokens? If so, this plugin might be for you.

UPDATE: The plugin was recently demoed at HashiCorp’s Hashitalks 2021.

Why?

Performing automation against GitHub APIs often neccessitates the creation of OAuth Tokens. These tokens are tied to a user account, have very coarsely-scoped permissions and do not expire.

As an organisation owner this likely means your automation-savvy users have created personal access tokens with powerful permissions which are being neither rotated nor deleted.

You will also commonly have wasted at least one of your GitHub seats on a robot/machine user for CI/CD purposes. These users share a similar access token and SSH key story as your human users do, and additonally need their credentials managed and rotated for them (arguably made more awkward when authenticating through an IdP).

GitHub Apps offer a better approach to this automation problem:

They do not consume a seat (license) nor need credential management.
They have much finer-grained permissions available to the access tokens.
The tokens they issue expire after one hour.

However, GitHub Apps require the management of at least one private key which is needed to mint the JWTs used for the App installation authentication token request flow.

What?

This plugin allows you to take advantage of your existing Vault deployment’s durable storage backend to protect the GitHub App private key, and your enabled Vault authN/Z mechanisms and highly-available API to perform the GitHub App token request flow on behalf of your users.

How?

The Vault plugin acts as an intermediary for a GitHub App that you install into your organisation:

Users login using your existing Vault auth backend(s) and, Vault RBAC permitting, can request GitHub tokens from the plugin. The plugin, in turn, authenticates with the GitHub App and requests a token on behalf of the user. This flow is better explained in the sequence diagram below:

Installation

To begin plugin installation, either download a release or build from source for your chosen OS and architecture.

From release

Always download the latest stable release from the releases section.

Verify

You can and should verify the authenticity and integrity of the plugin you downloaded. All released binaries are hashed and the resulting sums are signed by my GPG key.

# Import my key.
curl -sS https://github.com/martinbaillie.gpg | gpg --import -

# Verify the authenticity.
gpg --verify SHA256SUMS.sig SHA256SUMS

# Verify the integrity.
shasum -a 256 -c SHA256SUMS

From source

NOTE: You will need at least a Go 1.22+ toolchain to build this plugin from source. Ideally you will also be in the project’s Nix shell. See Development for more.

Either download the source zip/tar.gz of the latest release from the releases section and uncompress, or shallow clone to the target release tag as in below:
```
git clone --depth 1 -b <target_release_tag> \
    https://github.com/martinbaillie/vault-plugin-secrets-github.git
    
```

Build for your target OS and architecture.

# Recommended for accurate release-like reproduction.
goreleaser build --single-target # Your current OS/Arch.
GOOS=darwin GOARCH=amd64 goreleaser build --single-target # Another supported OS/Arch.
goreleaser build # All supported OS/Arch combinations.
# Alternatively,
go build
# or
nix build

Setup (GitHub)

NOTE: You will need access to admin of your GitHub or GitHub Enterprise organisation to continue with the one-time setup. You can of course use this plugin on your personal account if so desired.

Sign in as an org admin and begin creating a new GitHub App.
Choose any unique name, and homepage / webhook URLs (these can be anything; they are not required by the plugin).

NOTE: You may wish to take advantage of having GitHub call a webhook URL you own each time the App is used for auditing purposes.
Carefully choose the permissions the App will have access to. This is the superset of permissions. You will have the option of further restricting access to all or some repositories when you install an instance of the App, and users of this plugin will be able to even further restrict access in their individual token requests.
Decide if you want the app to be installable to other accounts. Usually you just want the one you are signed into.
Create the App. On the next screen GitHub will prompt you to create a Private Key. Do so, and save it somewhere safe (this is the key that will be ultimately lodged into the plugin configuration).
Note the App ID at the top of this page as well.

NOTE: Get the App ID anytime: Settings > Developer > settings > GitHub App > About item.
Click Install App from the LHS. You will be taken to the account installation pages where you can confirm the app was installed.
(OPTIONAL as of v1.3.0) Note the Installation ID from the URL of this page (usually: https://github.com/settings/installations/<installation id>) if you wish to configure using the installation ID directly.

NOTE: Get the Installation ID anytime: Settings > Developer > settings > GitHub Apps > Advanced > Payload in Request tab.

Setup (Vault)

Using the information noted from the previous step (Private Key, App ID and optionally the Installation ID), you are ready to move on to setting up the Vault plugin.

Move the desired plugin binary into your Vault’s configured plugin_directory.

mv vault-plugin-secrets-github-<os>-<arch> <plugin_directory>/vault-plugin-secrets-github

(OPTIONAL) Allow mlock() capabilities for the plugin binary. Memory locking is available to most UNIX-like OSes; the example below is for Linux.
```
setcap cap_ipc_lock=+ep <plugin_directory>/vault-plugin-secrets-github
    
```

(OPTIONAL) Calculate the SHA256 sum of the plugin and register it in Vault’s plugin catalog. If you are downloading the pre-compiled binary, it is highly recommended that you use the published SHA256SUMS file.

NOTE: The rest of these commands assume you have a valid VAULT_TOKEN and VAULT_API environment variables.

# If using a pre-compiled binary:
SHA256SUM=$(grep <downloaded_binary> SHA256SUMS | cut -d' ' -f1)
# If building from source:
SHA256SUM=$(shasum -a 256 <compiled_binary> | cut -d' ' -f1)

vault write sys/plugins/catalog/secret/vault-plugin-secrets-github \
    sha_256=${SHA256SUM} command=vault-plugin-secrets-github

Mount the secrets engine, choosing a prefix path (recommendation: github).

vault secrets enable -path=github -plugin-name=vault-plugin-secrets-github plugin

Configure the plugin with the details noted from the previous section.

# Write the configuration
vault write /github/config app_id=<app_id> prv_key=@<private_key_file>

# (OPTIONAL) Exclude repository metadata from token responses (reduces memory footprint).
vault write /github/config exclude_repository_metadata=true

# (OPTIONAL) Confirm the configuration landed as you expected.
vault read /github/config

# (OPTIONAL) Test a token creation.
vault read /github/token installation_id=<installation_id>
vault read /github/token org_name=<org_name> # Installation ID discovered from Org.

(OPTIONAL) Use Vault policy to constrain user capabilities on the GitHub endpoints. Example:

# Create a restrictive policy that only permits GitHub tokens that can write
# pull requests to a single repository.
vault policy write github-only-prs -<<EOF
path "github/token" {
  capabilities = ["update"]
  required_parameters = ["installation_id","permissions","repository_ids"]
  allowed_parameters = {
    "installation_id" = ["987654"]
    "repository_ids" = ["69857131"]
    "permissions"= ["pull_requests=write"]
  }
}
EOF

# Create and login as an example user with the policy attached.
vault auth enable userpass
vault write auth/userpass/users/martin password=baillie policies="github-only-prs"
vault login -method=userpass username=martin password=baillie

# Test the efficacy of the policy.
# Successfully creates token:
vault write /github/token installation_id=987654 repository_ids=69857131 permissions=pull_requests=write
# Permission denied:
vault write -f /github/token
vault write /github/token installation_id=987654 permissions=pull_requests=write
vault write /github/token installation_id=987654 repository_ids=69857131 permissions=pull_requests=read
vault write /github/token installation_id=987654 repository_ids=69857131 permissions=metadata=read
vault write /github/token installation_id=987654 repository_ids=123 permissions=pull_requests=write
vault write /github/token installation_id=987654 repository_ids=69857131
vault write /github/token installation_id=123456 repository_ids=69857131

API

Each plugin path is documented using Vault’s own help framework. To find out more information about any path, use vault path-help. For brevity, the API is also documented below.

Token

Instruct the plugin to create an installation access token against the configured GitHub App.

Method	Path	Produces
GET	/token	application/json
POST	/token	application/json
PUT	/token	application/json

Parameters

NOTE: Only one of installation_id or org_name is required. If only org_name is provided, an additional lookup against the GitHub instance is performed per token creation to discover the installation_id. If both are provided, installation_id takes precedence to avoid the additional round trip. Also note that no caching is performed so for high traffic use cases, favour installation_id.

All other parameters are optional. Omitting them results in a token that has access to all of the repositories and permissions that the GitHub App installation has.

When crafting Vault policy, hyper security sensitive organisations may wish to favour repository_ids (GitHub repository IDs are immutable) instead of repositories (GitHub repository names are mutable).

installation_id (int64) — the ID of the app installation.
org_name (string) — the organisation name.
repositories ([]string) — a list of the names of the repositories within the organisation that the installation token can access.
repository_ids ([]int64) — a list of the IDs of the repositories that the installation token can access. See this StackOverflow post for the quickest way to find a repository ID.
permissions (map[string]string) — a key value map of permission names to their access type (read or write). See GitHub’s documentation on permission names and access types.

Examples

# Create a token with all the superset of all permissions and repositories that
# the GitHub App installation has access to.
vault read /github/token

# Create a token with read permissions on the packages and write permissions on
# the pull requests of repositories named "demo-repo" and ID 123.
vault write /github/token \
  installation_id=456 \
	repository_ids=123 \
	repositories=demo-repo \
	permissions=packages=read \
	permissions=pull_requests=write

# Create a token with all permissions but only on the "demo-repo" repository.
vault write /github/token installation_id=456 repository_ids=123 repository_ids=456

# Create a token with all permissions but only on repositories 123 and 456.
vault write /github/token installation_id=456 repository_ids=123 repository_ids=456

# Create a token with write access to pull requests using read / GET.
vault write /github/token permissions=pull_requests=write

# Create a token with read access to metadata and write access to pull requests
# on repositories "demo-repo", 123 and 456 only.
# NOTE: Uses a Vault CLI JSON heredoc to submit the complex map type.
vault write /github/token - <<EOF
{
"installation_id": 456,
"repositories": ["demo-repo"],
"repository_ids": [123,456],
"permissions": {"metadata": "read", "pull_requests": "write"}
}
EOF

NOTE: a 422 response usually indicates you have requested repositories IDs or permissions that your GitHub App install does not have access to.

Revocation

It is possible to revoke tokens in your configured GitHub using Vault constructs. For example:

# Revoke an individual token with just the lease ID.
vault lease revoke <lease_id previously received from token endpoint>
# List currently active lease IDs.
vault list sys/leases/lookup/github/token
# Revoke all tokens currently leased by Vault.
vault lease revoke -prefix github/token

NOTE: the previous commands presume your plugin is mounted at /github.

Alternatively, you can go directly to your GitHub with the token in hand:

curl -H "Authorization: Bearer ${GITHUB_TOKEN}" \
	-X DELETE https://api.mygithub.com/installation/token

Permission sets

Instruct the plugin to create a specific permission set.

Method	Path	Produces
GET	/permissionset/<name>	application/json
POST	/permissionset/<name>	application/json
PUT	/permissionset/<name>	application/json
DELETE	/permissionset/<name>	application/json
GET	/permissionsets?list=true	application/json

Parameters

NOTE: Only one of installation_id or org_name is required. If only org_name is provided, an additional lookup against the GitHub instance is performed per token creation to discover the installation_id. If both are provided, installation_id takes precedence to avoid the additional round trip. Also note that no caching is performed so for high traffic use cases, favour installation_id.

All other parameters are optional. Omitting them results in a token that has access to all of the repositories and permissions that the GitHub App installation has.

When crafting Vault policy, hyper security sensitive organisations may wish to favour repository_ids (GitHub repository IDs are immutable) instead of repositories (GitHub repository names are mutable).

installation_id (int64) — the ID of the app installation.
org_name (string) — the organisation name.
repositories ([]string) — a list of the names of the repositories within the organisation that the installation token can access.
repository_ids ([]int64) — a list of the IDs of the repositories that the installation token can access. See this StackOverflow post for the quickest way to find a repository ID.
permissions (map[string]string) — a key value map of permission names to their access type (read or write). See GitHub’s documentation on permission names and access types.

Request a token from a permission set

Similar to the token flow in the previous section, you can instruct the plugin to create an installation access token by using a permission set name. The token returned will be constrained by that pre-configured permission set.

Method	Path	Produces
GET	/token/<name>	application/json
POST	/token/<name>	application/json
PUT	/token/<name>	application/json

Examples

# Configure a permission set that only allows metadata reads and PR writes
# against three repositories.
vault write /github/permissionset/demo-set - <<EOF
{
"installation_id": 987,
"repositories": ["demo-repo"],
"repository_ids": [123,456],
"permissions": {"metadata": "read", "pull_requests": "write"}
}
EOF
# Or
vault write /github/permissionset/demo-set \
	installation_id=987 \
	repositories=demo-repo \
	repository_ids=123 \
	repository_ids=456 \
	permissions=pull_requests=read \
	permissions=metadata=read

# Read the permission set configuration.
vault read /github/permissionset/demo-set

# List all permission sets.
vault list /github/permissionsets

# Create a token automatically constrained by the permission set.
vault read /github/token/demo-set

# Delete an existing permission set.
vault delete /github/permissionset/demo-set

Config

General CRUD operations against the configuration of the plugin.

Method	Path	Produces
POST	/config	application/json
GET	/config	application/json
PUT	/config	application/json
DELETE	/config	application/json

Parameters

app_id (int64) — the Application ID of the GitHub App.
prv_key (string) — a private key configured in the GitHub App. This private key must be in PEM PKCS#1 RSAPrivateKey format. It is not returned with read requests for security reasons but its presence or lack thereof is indicated.
base_url (string) — the base URL for API requests (defaults to the public GitHub API).
exclude_repository_metadata (bool) — reduce the verbose `repositories` array in GitHub token responses to a simple list of repository names. This significantly reduces the memory required by the plugin when used at scale.

Examples

# Write the plugin configuration using the default base URL, and reading the key from a file.
vault write /github/config app_id=123 [email protected]

# Read the plugin configuration.
vault read /github/config

# Update the plugin configuration to a GitHub Enterprise base URL.
vault write /github/config base_url="https://api.mygithub.org"

# Significantly reduce memory consumed per token.
vault write /github/config exclude_repository_metadata=true

# Delete the plugin configuration.
vault delete /github/config

Metrics

Prometheus/OpenMetrics formatted metrics exposition.

Method	Path	Produces
GET	/metrics	text/plain

Metrics

In addition to standard Go metrics, the following custom metrics are exposed:

vault_github_token_request_duration_seconds — a summary of token request latency and status.
vault_github_token_revocation_request_duration_seconds — a summary of token revocation request latency and status.
vault_github_token_build_info — a constant with useful build information.

Sample Dashboard

A sample dashboard is provided.

Info

Information about the GitHub secrets plugin, such as the plugin version, VCS detail and where to get help.

Method	Path	Produces
GET	/info	application/json

Development

This plugin is written using modern Go (1.22+). Its project infrastructure supports Linux and macOS aarch64/amd64 platforms and is handled by Nix.

Nix is the only pre-requisite you need to have installed. If you do not already have or wish to install nix, consider the nixos/nix container image.

The Nix flake at the project root provides the Go toolchain as well as any ancillary tooling needed. If you use direnv (recommended) you will automatically be in a usable project shell after trusting it once with direnv allow.

Otherwise you will need to enter a Nix shell manually:

nix develop

Once in the Nix shell you should be presented with the following:

vault-plugin-secrets-github
menu                              - available commands

Proceed to browse the menu or learn how to test the project below.

Tests

This plugin is comprehensively tested by both unit and acceptance tests. Pull requests that do not maintain an >90% coverage will not be accepted.

# View the developer shell menu.
menu

# Run linting.
lint

# Run CI-grade linting.
env CI=true lint

# Run unit tests.
unit

# Run unit and acceptance tests, integrating against a local Vault and stubbed
# GitHub API.
integration

# Run integration tests with debug logging.
env DEBUG=true integration

# Run unit and acceptance tests, integrating against a local Vault and the real
# GitHub API against your own GitHub App installation (you can also run these
# tests against a GitHub Enterprise deployment). Setting `BASE_URL` causes the
# tests to call against a real API.
# NOTE: Keep in mind this test configuration will create real tokens.
env \
    BASE_URL=https://api.github.com \
    APP_ID=<your application id> \
    ORG_NAME=<org_name> \
    INSTALLATION_ID=<installation_id> \
    PRV_KEY="$(cat /path/to/your/app/prv_key_file)" integration

Security

HashiCorp and GitHub take their security seriously. If you believe you have found a security issue with either through using this plugin, do not open an issue here. Responsibly disclose by getting in touch with HashiCorp or GitHub security teams respectively.

Name		Name	Last commit message	Last commit date
Latest commit History 76 Commits
.github		.github
github		github
.editorconfig		.editorconfig
.envrc		.envrc
.gitignore		.gitignore
.golangci.yml		.golangci.yml
.goreleaser.yml		.goreleaser.yml
LICENSE		LICENSE
README.org		README.org
dashboard.json		dashboard.json
dashboard.png		dashboard.png
flake.lock		flake.lock
flake.nix		flake.nix
go.mod		go.mod
go.sum		go.sum
gomod2nix.toml		gomod2nix.toml
main.go		main.go
vault_github_plugin.png		vault_github_plugin.png

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

Vault Plugin Secrets GitHub

About

Why?

What?

How?

Installation

From release

Verify

From source

Setup (GitHub)

Setup (Vault)

API

Token

Parameters

Examples

Revocation

Permission sets

Parameters

Request a token from a permission set

Examples

Config

Parameters

Examples

Metrics

Metrics

Sample Dashboard

Info

Development

Tests

Security

About

Releases 9

Sponsor this project

Packages

Contributors 10

Languages

License

martinbaillie/vault-plugin-secrets-github

Folders and files

Latest commit

History

Repository files navigation

Vault Plugin Secrets GitHub

About

Why?

What?

How?

Installation

From release

Verify

From source

Setup (GitHub)

Setup (Vault)

API

Token

Parameters

Examples

Revocation

Permission sets

Parameters

Request a token from a permission set

Examples

Config

Parameters

Examples

Metrics

Metrics

Sample Dashboard

Info

Development

Tests

Security

About

Topics

Resources

License

Stars

Watchers

Forks

Releases 9

Sponsor this project

Packages 0

Contributors 10

Languages

Packages