Skip to content

Commit

Permalink
Merge branch 'current' into notices
Browse files Browse the repository at this point in the history
  • Loading branch information
matthewshaver authored Jan 8, 2024
2 parents a3e8e2a + 99a4b47 commit 3244267
Show file tree
Hide file tree
Showing 25 changed files with 613 additions and 69 deletions.
18 changes: 11 additions & 7 deletions website/docs/best-practices/how-we-mesh/mesh-1-intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,19 +6,21 @@ hoverSnippet: Learn how to get started with dbt Mesh

## What is dbt Mesh?

Organizations of all sizes rely upon dbt to manage their data transformations, from small startups to large enterprises. At scale, it can be challenging to coordinate all the organizational and technical requirements demanded by your stakeholders within the scope of a single dbt project. To date, there also hasn't been a first-class way to effectively manage the dependencies, governance, and workflows between multiple dbt projects.
Organizations of all sizes rely upon dbt to manage their data transformations, from small startups to large enterprises. At scale, it can be challenging to coordinate all the organizational and technical requirements demanded by your stakeholders within the scope of a single dbt project.

Regardless of your organization's size and complexity, dbt should empower data teams to work independently and collaboratively; sharing data, code, and best practices without sacrificing security or autonomy. dbt Mesh provides the tooling for teams to finally achieve this.
To date, there also hasn't been a first-class way to effectively manage the dependencies, governance, and workflows between multiple dbt projects.

dbt Mesh is not a single product: it is a pattern enabled by a convergence of several features in dbt:
That's where **dbt Mesh** comes in - empowering data teams to work *independently and collaboratively*; sharing data, code, and best practices without sacrificing security or autonomy.

This guide will walk you through the concepts and implementation details needed to get started. dbt Mesh is not a single product - it is a pattern enabled by a convergence of several features in dbt:

- **[Cross-project references](/docs/collaborate/govern/project-dependencies#how-to-write-cross-project-ref)** - this is the foundational feature that enables the multi-project deployments. `{{ ref() }}`s now work across dbt Cloud projects on Enterprise plans.
- **[dbt Explorer](/docs/collaborate/explore-projects)** - dbt Cloud's metadata-powered documentation platform, complete with full, cross-project lineage.
- **Governance** - dbt's new governance features allow you to manage access to your dbt models both within and across projects.
- **[Groups](/docs/collaborate/govern/model-access#groups)** - groups allow you to assign models to subsets within a project.
- **Governance** - dbt's governance features allow you to manage access to your dbt models both within and across projects.
- **[Groups](/docs/collaborate/govern/model-access#groups)** - With groups, you can organize nodes in your dbt DAG that share a logical connection (for example, by functional area) and assign an owner to the entire group.
- **[Access](/docs/collaborate/govern/model-access#access-modifiers)** - access configs allow you to control who can reference models.
- **[Model Versions](/docs/collaborate/govern/model-versions)** - when coordinating across projects and teams, we recommend treating your data models as stable APIs. Model versioning is the mechanism to allow graceful adoption and deprecation of models as they evolve.
- **[Model Contracts](/docs/collaborate/govern/model-contracts)** - data contracts set explicit expectations on the shape of the data to ensure data changes upstream of dbt or within a project's logic don't break downstream consumers' data products.
- **[Model Versions](/docs/collaborate/govern/model-versions)** - when coordinating across projects and teams, we recommend treating your data models as stable APIs. Model versioning is the mechanism to allow graceful adoption and deprecation of models as they evolve.
- **[Model Contracts](/docs/collaborate/govern/model-contracts)** - data contracts set explicit expectations on the shape of the data to ensure data changes upstream of dbt or within a project's logic don't break downstream consumers' data products.

## Who is dbt Mesh for?

Expand All @@ -32,6 +34,8 @@ dbt Cloud is designed to coordinate the features above and simplify the complexi

If you're just starting your dbt journey, don't worry about building a multi-project architecture right away. You can _incrementally_ adopt the features in this guide as you scale. The collection of features work effectively as independent tools. Familiarizing yourself with the tooling and features that make up a multi-project architecture, and how they can apply to your organization will help you make better decisions as you grow.

For additional information, refer to the [dbt Mesh FAQs](/best-practices/how-we-mesh/mesh-4-faqs).

## Learning goals

- Understand the **purpose and tradeoffs** of building a multi-project architecture.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -127,4 +127,4 @@ We've provided a set of example projects you can use to explore the topics cover

### dbt-meshify

We recommend using the `dbt-meshify` [command line tool](<https://dbt-labs.github.io/dbt-meshify/0.1/>) to help you do this. This comes with CLI operations to automate most of the above steps.
We recommend using the `dbt-meshify` [command line tool](<https://dbt-labs.github.io/dbt-meshify/>) to help you do this. This comes with CLI operations to automate most of the above steps.
Loading

0 comments on commit 3244267

Please sign in to comment.