Skip to content

Commit

Permalink
This branch was auto-updated!
Browse files Browse the repository at this point in the history
  • Loading branch information
github-actions[bot] authored Nov 29, 2023
2 parents d6da5fd + 615c24e commit 8b63bdd
Show file tree
Hide file tree
Showing 3 changed files with 41 additions and 25 deletions.
10 changes: 2 additions & 8 deletions website/docs/docs/build/packages.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,15 +25,9 @@ dbt _packages_ are in fact standalone dbt projects, with models and macros that
* It's important to note that defining and installing dbt packages is different from [defining and installing Python packages](/docs/build/python-models#using-pypi-packages)


:::info `dependencies.yml` has replaced `packages.yml`
Starting from dbt v1.6, `dependencies.yml` has replaced `packages.yml`. This file can now contain both types of dependencies: "package" and "project" dependencies.
- "Package" dependencies lets you add source code from someone else's dbt project into your own, like a library.
- "Project" dependencies provide a different way to build on top of someone else's work in dbt. Refer to [Project dependencies](/docs/collaborate/govern/project-dependencies) for more info.
-
You can rename `packages.yml` to `dependencies.yml`, _unless_ you need to use Jinja within your packages specification. This could be necessary, for example, if you want to add an environment variable with a git token in a private git package specification.

:::
import UseCaseInfo from '/snippets/_packages_or_dependencies.md';

<UseCaseInfo/>

## How do I add a package to my project?
1. Add a file named <VersionBlock firstVersion="1.6"> `dependencies.yml` or </VersionBlock> `packages.yml` to your dbt project. This should be at the same level as your `dbt_project.yml` file.
Expand Down
22 changes: 5 additions & 17 deletions website/docs/docs/collaborate/govern/project-dependencies.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,12 @@ This year, dbt Labs is introducing an expanded notion of `dependencies` across m
- **Packages** &mdash; Familiar and pre-existing type of dependency. You take this dependency by installing the package's full source code (like a software library).
- **Projects** &mdash; A _new_ way to take a dependency on another project. Using a metadata service that runs behind the scenes, dbt Cloud resolves references on-the-fly to public models defined in other projects. You don't need to parse or run those upstream models yourself. Instead, you treat your dependency on those models as an API that returns a dataset. The maintainer of the public model is responsible for guaranteeing its quality and stability.

import UseCaseInfo from '/snippets/_packages_or_dependencies.md';

<UseCaseInfo/>

Refer to the [FAQs](#faqs) for more info.

Starting in dbt v1.6 or higher, `packages.yml` has been renamed to `dependencies.yml`. However, if you need use Jinja within your packages config, such as an environment variable for your private package, you need to keep using `packages.yml` for your packages for now. Refer to the [FAQs](#faqs) for more info.

## Prerequisites

Expand All @@ -33,22 +37,6 @@ In order to add project dependencies and resolve cross-project `ref`, you must:
- Have a successful run of the upstream ("producer") project
- Have a multi-tenant or single-tenant [dbt Cloud Enterprise](https://www.getdbt.com/pricing) account (Azure ST is not supported but coming soon)

<!-- commenting out in case we can repurpose content
### About dependencies.yml
There are some important differences between using a `dependencies.yml` compared to a `packages.yml` file:
- `dependencies.yml`
- Primarily designed for dbt Mesh and cross-project reference workflow.
- Supports both Projects and non-private dbt packages (private packages aren't supported yet).
- Helps maintain your project's organization by allowing you to specify hub packages like `dbt_utils`, reducing the need for multiple YAML files.
- Does not support conditional configuration using Jinja-in-yaml
- `packages.yml`
- Does not contribute to the dbt Mesh workflow.
- Serves as a list of dbt Packages (such as dbt projects) that you want to download into your root or parent dbt project.
- Can only include packages, including private packages (doesn't support Projects)
-->
## Example

As an example, let's say you work on the Marketing team at the Jaffle Shop. The name of your team's project is `jaffle_marketing`:
Expand Down
34 changes: 34 additions & 0 deletions website/snippets/_packages_or_dependencies.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@

## Use cases

Starting from dbt v1.6, `dependencies.yml` has replaced `packages.yml`. The `dependencies.yml` file can now contain both types of dependencies: "package" and "project" dependencies.
- ["Package" dependencies](/docs/build/packages) lets you add source code from someone else's dbt project into your own, like a library.
- ["Project" dependencies](/docs/collaborate/govern/project-dependencies) provide a different way to build on top of someone else's work in dbt.

If your dbt project doesn't require the use of Jinja within the package specifications, you can simply rename your existing `packages.yml` to `dependencies.yml`. However, something to note is if your project's package specifications use Jinja, particularly for scenarios like adding an environment variable or a [Git token method](/docs/build/packages#git-token-method) in a private Git package specification, you should continue using the `packages.yml` file name.

There are some important differences between Package dependencies and Project dependencies:

<Tabs>
<TabItem value="project" label="When to use Project dependencies">

Project dependencies are designed for the [dbt Mesh](/best-practices/how-we-mesh/mesh-1-intro) and [cross-project reference](/docs/collaborate/govern/project-dependencies#how-to-use-ref) workflow:

- Use `dependencies.yml` when you need to set up cross-project references between different dbt projects, especially in a dbt Mesh setup.
- Use `dependencies.yml` when you want to include both projects and non-private dbt packages in your project's dependencies.
- Private packages are not supported in `dependencies.yml` because they intentionally don't support Jinja rendering or conditional configuration. This is to maintain static and predictable configuration and ensures compatibility with other services, like dbt Cloud.
- Use `dependencies.yml` for organization and maintainability. It can help maintain your project's organization by allowing you to specify [dbt Hub packages](https://hub.getdbt.com/) like `dbt_utils`. This reduces the need for multiple YAML files to manage dependencies.

</TabItem>

<TabItem value="packages" label="When to use Package dependencies">

Package dependencies allow you to add source code from someone else's dbt project into your own, like a library:

- Use `packages.yml` when you want to download dbt packages, such as dbt projects, into your root or parent dbt project. Something to note is that it doesn't contribute to the dbt Mesh workflow.
- Use `packages.yml` to include packages, including private packages, in your project's dependencies. If you have private packages that you need to reference, `packages.yml` is the way to go.
- `packages.yml` supports Jinja rendering for historical reasons, allowing dynamic configurations. This can be useful if you need to insert values, like a [Git token method](/docs/build/packages#git-token-method) from an environment variable, into your package specifications.

Currently, to use private git repositories in dbt, you need to use a workaround that involves embedding a git token with Jinja. This is not ideal as it requires extra steps like creating a user and sharing a git token. We're planning to introduce a simpler method soon that won't require Jinja-embedded secret environment variables. For that reason, `dependencies.yml` does not support Jinja.
</TabItem>
</Tabs>

0 comments on commit 8b63bdd

Please sign in to comment.