Skip to content

Commit

Permalink
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Private packages beta
Browse files Browse the repository at this point in the history
matthewshaver committed Dec 16, 2024
1 parent bc41a7e commit f90ef93
Showing 1 changed file with 66 additions and 4 deletions.
70 changes: 66 additions & 4 deletions website/docs/docs/build/packages.md
Original file line number Diff line number Diff line change
@@ -157,9 +157,64 @@ packages:

Where `name: 'dbt_utils'` specifies the subfolder of `dbt_packages` that's created for the package source code to be installed within.

### Private packages
## Private packages

#### SSH Key Method (Command Line only)
### Native private packages <Lifecycle status='beta'/>

dbt Cloud supports private packages from [supported](#prerequisites) Git repos leveraging an exisiting [configuration](/docs/cloud/git/git-configuration-in-dbt-cloud) in your environment. Previously, you had to configure a [token](#git-token-method) to retrieve packages from your private repos.

#### Prerequisites

Native support for private packages is supported on the following services:
- [Github](/docs/cloud/git/connect-github)
- [Azure DevOps](/docs/cloud/git/connect-azure-devops)

Support for Gitlab is coming soon.

You must have the Git service configured in the **Integrations** section of your **Account settings** to use this feature.

#### Configuration

Use the `private` block in your `packages.yml` or `dependencies.yml` to clone package repos using your existing dbt Cloud Git integration without having to provision an access token or create a dbt Cloud environment variable:

<File name="packages.yml">

```yaml
packages:
- private: dbt-labs/awesome_repo
- package: normal packages
[...]
```

</File>

You can pin private packages similar to regular dbt packages:

```yaml
packages:
- private: dbt-labs/awesome_repo
revision: branch name | tagged git release | a specific commit (full 40-character hash}
```

If you are using multiple Git integrations, disambiguate by adding the provider key:

```yaml
packages:
- private: dbt-labs/awesome_repo
provider: "github" | "azure"
```

With this method, you can retrieve private packages from an integrated Git provider without any additional steps to connect.

### SSH Key Method (Command Line only)
If you're using the Command Line, private packages can be cloned via SSH and an SSH key.

When you use SSH keys to authenticate to your git remote server, you don’t need to supply your username and password each time. Read more about SSH keys, how to generate them, and how to add them to your git provider here: [Github](https://docs.github.com/en/github/authenticating-to-github/connecting-to-github-with-ssh) and [GitLab](https://docs.gitlab.com/ee/user/ssh.html).
@@ -177,7 +232,14 @@ packages:
If you're using dbt Cloud, the SSH key method will not work, but you can use the [HTTPS Git Token Method](https://docs.getdbt.com/docs/build/packages#git-token-method).


#### Git token method
### Git token method

:::note

dbt Cloud has [native support](#native-private-packages) for Git hosted private packages for Github and Azure DevOps (GitLab coming soon). If you are using a supported [integrated Git environment](/docs/cloud/git/git-configuration-in-dbt-cloud), you no longer need to configure Git tokens to retrieve private packages.

:::

This method allows the user to clone via HTTPS by passing in a git token via an environment variable. Be careful of the expiration date of any token you use, as an expired token could cause a scheduled run to fail. Additionally, user tokens can create a challenge if the user ever loses access to a specific repo.


@@ -246,7 +308,7 @@ Read more about creating a Personal Access Token [here](https://confluence.atlas



#### Configure subdirectory for packaged projects
## Configure subdirectory for packaged projects

In general, dbt expects `dbt_project.yml` to be located as a top-level file in a package. If the packaged project is instead nested in a subdirectory—perhaps within a much larger mono repo—you can optionally specify the folder path as `subdirectory`. dbt will attempt a [sparse checkout](https://git-scm.com/docs/git-sparse-checkout) of just the files located within that subdirectory. Note that you must be using a recent version of `git` (`>=2.26.0`).

0 comments on commit f90ef93

Please sign in to comment.