diff --git a/0.5/commands/index.html b/0.5/commands/index.html index 30df624..ad730e9 100644 --- a/0.5/commands/index.html +++ b/0.5/commands/index.html @@ -481,9 +481,11 @@

dbt-meshify operation bump-version --selector TUPLE The name(s) of the YML selector specifying the resources to include in the operation -l, --latest Makes the newest version the latest version when - incrementing model versions - --defined-in TEXT - --help Show this message and exit. + creating model versions + --increment, --i Increments the latest_version setting by 1 when + creating model versions + --defined-in TEXT + --help Show this message and exit.

dbt-meshify operation create-group#

Create a group and add selected resources to the group.

@@ -546,9 +548,11 @@

dbt-meshify version --selector TUPLE The name(s) of the YML selector specifying the resources to include in the operation -l, --latest Makes the newest version the latest version when - incrementing model versions - --defined-in TEXT - --help Show this message and exit. + creating model versions + --increment, --i Increments the latest_version setting by 1 when + creating model versions + --defined-in TEXT + --help Show this message and exit. diff --git a/0.5/search/search_index.json b/0.5/search/search_index.json index f686800..d457f8c 100644 --- a/0.5/search/search_index.json +++ b/0.5/search/search_index.json @@ -1 +1 @@ -{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"dbt_meshify","text":"

dbt-meshify is a CLI tool that automates the creation of model governance and cross-project lineage features introduced in dbt-core v1.5 and v1.6. This package will leverage your dbt project metadata to create and/or edit the files in your project to properly configure the models in your project with these features.

These features include:

  1. Groups - group your models into logical sets.
  2. Contracts - add model contracts to your models to ensure consistent data shape.
  3. Access - control the access level of models within groups
  4. Versions - create and increment versions of particular models.
  5. Project dependencies - split a monolithic dbt project into component projects, or connect multiple pre-existing dbt projects using cross-project ref.

This package leverages the dbt-core Python API to allow users to use standard dbt selection syntax for each of the commands in this package (unless otherwise noted). See details on each of the specific commands available on the commands page.

"},{"location":"#getting-started","title":"Getting Started","text":"

This package helps automate the code development required for adding the model governance and cross-project lineage features mentioned above.

The first question to ask yourself is \"which of these features do I want to add to my project\"? Do you want to add contracts, create a new group, split your monolithic dbt project in two? Your answer to this question will establish which dbt-meshify command is right for you!

This package consists of component and global commands - so you can decide how to best break apart your work.

The component commands allow you to do a single step at a time and begin with dbt-meshify operation. For example, if you wanted to add a new version to a model, you would run something like dbt-meshify operation add-version --select fct_orders. This command would:

  1. add version configuration values to fct_orders

and that's it!

The global commands combine multiple component commands to complete a larger set of work and begin with dbt-meshify. For example, if you wanted to define a group for a subset of your models, you would run something like dbt-meshify group finance --owner-name \"Monopoly Man\" --select +tag:finance. This command would:

  1. define a new group named \"finance\" in your dbt project, setting the owner name to \"Monopoly Man\"
  2. add all models tagged with \"finance\" to that new group
  3. set access to protected for all \"leaf\" models (models with no downstream dependencies) and models with cross-group dependencies
  4. add contracts to all protected nodes

all at once!

The next question to ask yourself is \"which of my models do I want to add these features to?\". This informs the selection syntax you provide to the dbt-meshify command of choice. dbt-meshify uses the same selection syntax as dbt, so you can use the --select, --exclude and --selector flags to select resources based on model names, tags, and so on!

Once you've decided:

  1. which feature(s) you want to add to your dbt project
  2. which subset of models you want to add those feature(s) to

you're ready to use dbt-meshify!

For further information, check out the available commands or read through some examples.

"},{"location":"#what-dbt-meshify-does-not-handle","title":"What dbt-meshify does not handle","text":"

There are a handful of known edge cases that this package does not automatically handle. In these cases, we recommend doing a manual check to make sure you've handled these appropriately:

edge case manual check dbt-meshify split copies over the entire contents of the packages.yml file from the original project to the new subproject remove unnecessary packages from each project dbt-meshify split makes a copy of all necessary macros from the original project to the new subproject consider creating a private \"macros only\" project to install as a package into all of your other projects, instead of maintaining duplicate copies of shared macros"},{"location":"commands/","title":"dbt-meshify Commands","text":""},{"location":"commands/#dbt-meshify","title":"dbt-meshify","text":"

Usage:

dbt-meshify [OPTIONS] COMMAND [ARGS]...\n

Options:

  --dry-run\n  --debug\n  --version  Show version information and exit\n  --help     Show this message and exit.\n

Subcommands

"},{"location":"commands/#dbt-meshify-connect","title":"dbt-meshify connect","text":"

Connects multiple dbt projects together by adding all necessary dbt Mesh constructs

Usage:

dbt-meshify connect [OPTIONS]\n

Options:

  --project-paths TUPLE         The paths to the set of dbt projects to\n                                connect. Must supply 2+ paths.\n  --projects-dir PATH           The path to a directory containing multiple\n                                dbt projects. Directory must contain 2+\n                                projects.\n  -e, --exclude-projects TUPLE  The set of dbt projects to exclude from the\n                                operation when using the --projects-dir\n                                option.\n  -r, --read-catalog            Skips the dbt docs generate step and reads the\n                                local catalog.json file.\n  --help                        Show this message and exit.\n
"},{"location":"commands/#dbt-meshify-group","title":"dbt-meshify group","text":"

Creates a new dbt group based on the selection syntax Detects the edges of the group, makes their access protected, and adds contracts to them

Usage:

dbt-meshify group [OPTIONS] NAME\n

Options:

  -e, --exclude TUPLE      The dbt selection syntax specifying the resources\n                           to exclude in the operation\n  --group-yml-path PATH    An optional path to store the new group YAML\n                           definition.\n  --owner-email TEXT       The group Owner's email address.\n  --owner-name TEXT        The group Owner's name.\n  --owner-properties TEXT  Additional properties to assign to a group Owner.\n  --project-path PATH      The path to the dbt project to operate on. Defaults\n                           to the current directory.\n  -r, --read-catalog       Skips the dbt docs generate step and reads the\n                           local catalog.json file.\n  -s, --select TUPLE       The dbt selection syntax specifying the resources\n                           to include in the operation\n  --selector TUPLE         The name(s) of the YML selector specifying the\n                           resources to include in the operation\n  --help                   Show this message and exit.\n
"},{"location":"commands/#dbt-meshify-operation","title":"dbt-meshify operation","text":"

Set of subcommands for performing mesh operations on dbt projects

Usage:

dbt-meshify operation [OPTIONS] COMMAND [ARGS]...\n

Options:

  --help  Show this message and exit.\n

Subcommands

"},{"location":"commands/#dbt-meshify-operation-add-contract","title":"dbt-meshify operation add-contract","text":"

Adds a contract to all selected models.

Usage:

dbt-meshify operation add-contract [OPTIONS]\n

Options:

  -e, --exclude TUPLE  The dbt selection syntax specifying the resources to\n                       exclude in the operation\n  --project-path PATH  The path to the dbt project to operate on. Defaults to\n                       the current directory.\n  -r, --read-catalog   Skips the dbt docs generate step and reads the local\n                       catalog.json file.\n  -s, --select TUPLE   The dbt selection syntax specifying the resources to\n                       include in the operation\n  --selector TUPLE     The name(s) of the YML selector specifying the\n                       resources to include in the operation\n  --help               Show this message and exit.\n
"},{"location":"commands/#dbt-meshify-operation-add-version","title":"dbt-meshify operation add-version","text":"

Adds model version boilerplate for all selected models.

Usage:

dbt-meshify operation add-version [OPTIONS]\n

Options:

  -e, --exclude TUPLE  The dbt selection syntax specifying the resources to\n                       exclude in the operation\n  --project-path PATH  The path to the dbt project to operate on. Defaults to\n                       the current directory.\n  -r, --read-catalog   Skips the dbt docs generate step and reads the local\n                       catalog.json file.\n  -s, --select TUPLE   The dbt selection syntax specifying the resources to\n                       include in the operation\n  --selector TUPLE     The name(s) of the YML selector specifying the\n                       resources to include in the operation\n  --defined-in TEXT\n  --help               Show this message and exit.\n
"},{"location":"commands/#dbt-meshify-operation-bump-version","title":"dbt-meshify operation bump-version","text":"

Create new model versions for all selected models.

Usage:

dbt-meshify operation bump-version [OPTIONS]\n

Options:

  -e, --exclude TUPLE  The dbt selection syntax specifying the resources to\n                       exclude in the operation\n  --project-path PATH  The path to the dbt project to operate on. Defaults to\n                       the current directory.\n  -r, --read-catalog   Skips the dbt docs generate step and reads the local\n                       catalog.json file.\n  -s, --select TUPLE   The dbt selection syntax specifying the resources to\n                       include in the operation\n  --selector TUPLE     The name(s) of the YML selector specifying the\n                       resources to include in the operation\n  -l, --latest         Makes the newest version the latest version when\n                       incrementing model versions\n  --defined-in TEXT\n  --help               Show this message and exit.\n
"},{"location":"commands/#dbt-meshify-operation-create-group","title":"dbt-meshify operation create-group","text":"

Create a group and add selected resources to the group.

Usage:

dbt-meshify operation create-group [OPTIONS] NAME\n

Options:

  -e, --exclude TUPLE      The dbt selection syntax specifying the resources\n                           to exclude in the operation\n  --group-yml-path PATH    An optional path to store the new group YAML\n                           definition.\n  --owner-email TEXT       The group Owner's email address.\n  --owner-name TEXT        The group Owner's name.\n  --owner-properties TEXT  Additional properties to assign to a group Owner.\n  --project-path PATH      The path to the dbt project to operate on. Defaults\n                           to the current directory.\n  -r, --read-catalog       Skips the dbt docs generate step and reads the\n                           local catalog.json file.\n  -s, --select TUPLE       The dbt selection syntax specifying the resources\n                           to include in the operation\n  --selector TUPLE         The name(s) of the YML selector specifying the\n                           resources to include in the operation\n  --help                   Show this message and exit.\n
"},{"location":"commands/#dbt-meshify-split","title":"dbt-meshify split","text":"

Splits out a new subproject from a dbt project by adding all necessary dbt Mesh constructs to the resources based on the selected resources.

Usage:

dbt-meshify split [OPTIONS] PROJECT_NAME\n

Options:

  --create-path PATH   The path to create the new dbt project. Defaults to the\n                       name argument supplied.\n  -e, --exclude TUPLE  The dbt selection syntax specifying the resources to\n                       exclude in the operation\n  --project-path PATH  The path to the dbt project to operate on. Defaults to\n                       the current directory.\n  -r, --read-catalog   Skips the dbt docs generate step and reads the local\n                       catalog.json file.\n  -s, --select TUPLE   The dbt selection syntax specifying the resources to\n                       include in the operation\n  --selector TUPLE     The name(s) of the YML selector specifying the\n                       resources to include in the operation\n  --help               Show this message and exit.\n
"},{"location":"commands/#dbt-meshify-version","title":"dbt-meshify version","text":"

Increment the models to the next version, and create in the initial version if it has not yet been defined.

Usage:

dbt-meshify version [OPTIONS]\n

Options:

  -e, --exclude TUPLE  The dbt selection syntax specifying the resources to\n                       exclude in the operation\n  --project-path PATH  The path to the dbt project to operate on. Defaults to\n                       the current directory.\n  -r, --read-catalog   Skips the dbt docs generate step and reads the local\n                       catalog.json file.\n  -s, --select TUPLE   The dbt selection syntax specifying the resources to\n                       include in the operation\n  --selector TUPLE     The name(s) of the YML selector specifying the\n                       resources to include in the operation\n  -l, --latest         Makes the newest version the latest version when\n                       incrementing model versions\n  --defined-in TEXT\n  --help               Show this message and exit.\n
"},{"location":"examples/","title":"Examples","text":"

For consistency and clarity of the following examples, we're going to use a simplified dbt project. In practice, the model governance and cross-project lineage features describe are most beneficial for large dbt projects that are struggling to scale.

We will give a basic example for each command, but to see the full list of additional flags you can add to a given command, check out the commands page.

Note

One helpful flag that you can add to all of the commands is --read-catalog, which will skip the dbt docs generate step and instead read the local catalog.json file - this will speed up the time it takes to run the dbt-meshify commands but relies on your local catalog.json file being up-to-date. Alternatively, you can configure this via the DBT_MESHIFY_READ_CATALOG environment variable.

Let's imagine a dbt project with the following models:

You can checkout the source code for this example here.

"},{"location":"examples/#component-commands","title":"Component commands","text":""},{"location":"examples/#create-a-new-group","title":"Create a new group","text":"

Let's say you want to create a new group for your sales analytics models.

You can run the following command:

dbt-meshify operation create-group sales_analytics --owner-name Ralphie --select +int_sales__unioned +int_returns__unioned transactions\n

This will create a new group named \"sales_analytics\" with the owner \"Ralphie\" and add all selected models to that group with the appropriate access configuration:

"},{"location":"examples/#addincrement-model-versions","title":"Add/increment model versions","text":"

Let's say you want to add a new version to the customers model, which is currently un-versioned. Versions can provide a smoother upgrade pathway when introducing breaking changes to models that have downstream dependencies.

You can run the following command:

dbt-meshify version --select customers\n

This will add a version to the customers model for your current version, and will add a new version for breaking change you wish to implement:

"},{"location":"examples/#add-contracts","title":"Add contract(s)","text":"

Let's say you want to add a new contract to the stores model, which is currently un-contracted.

You can run the following command:

dbt-meshify operation add-contract --select stores\n

This will add an enforced contract to the stores model:

"},{"location":"examples/#global-commands","title":"Global commands","text":""},{"location":"examples/#group-together-a-subset-of-models","title":"Group together a subset of models","text":"

Let's say you want to group together your sales analytics models - create a new group and add contracts to appropriate models simultaneously.

You can run the following command:

dbt-meshify group sales_analytics --owner-name Ralphie --select +int_sales__unioned +int_returns__unioned transactions\n

This will create a new group named \"sales_analytics\" with the owner \"Ralphie\", add all selected models to that group with the appropriate access configuration, _and add contracts to the models at the boundary between this group and the rest of the project__:

"},{"location":"examples/#split-out-a-new-subproject","title":"Split out a new subproject","text":"

Let's say you want to split our your sales analytics models into a new subproject.

You can run the following command:

dbt-meshify split sales_analytics --select +int_sales__unioned +int_returns__unioned transactions\n

This will create a new subproject that contains the selected sales analytics models, configure the \"edge\" models to be public and contracted, and replace all dependencies in the downstream project on the upstreams's models with cross-project refs:

By default, the new subproject will be created in the current directory; however, you can use the --create-path flag to create it in any directory you like.

"},{"location":"examples/#connect-multiple-dbt-projects","title":"Connect multiple dbt projects","text":"

Let's look at a slightly modified version of the example we've been working with. Instead of a single dbt project, let's imagine you're starting with two separate dbt projects connected via the \"source hack\":

We call this type of multi-project configuration the \"source hack\" because there are models generated by project A (stores and customers) that are defined as sources in project B.

Let's say we want to connect these two projects using model governance best practices and cross project refs.

You can run the following command:

dbt-meshify connect --project-paths path/to/project_a path/to/project_b\n

This will make the upstream project a dependency for the downstream project, configure the \"edge\" models to be public and contracted, and replace all dependencies in the downstream project on the upstreams's models with cross-project refs:

"}]} \ No newline at end of file +{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"dbt_meshify","text":"

dbt-meshify is a CLI tool that automates the creation of model governance and cross-project lineage features introduced in dbt-core v1.5 and v1.6. This package will leverage your dbt project metadata to create and/or edit the files in your project to properly configure the models in your project with these features.

These features include:

  1. Groups - group your models into logical sets.
  2. Contracts - add model contracts to your models to ensure consistent data shape.
  3. Access - control the access level of models within groups
  4. Versions - create and increment versions of particular models.
  5. Project dependencies - split a monolithic dbt project into component projects, or connect multiple pre-existing dbt projects using cross-project ref.

This package leverages the dbt-core Python API to allow users to use standard dbt selection syntax for each of the commands in this package (unless otherwise noted). See details on each of the specific commands available on the commands page.

"},{"location":"#getting-started","title":"Getting Started","text":"

This package helps automate the code development required for adding the model governance and cross-project lineage features mentioned above.

The first question to ask yourself is \"which of these features do I want to add to my project\"? Do you want to add contracts, create a new group, split your monolithic dbt project in two? Your answer to this question will establish which dbt-meshify command is right for you!

This package consists of component and global commands - so you can decide how to best break apart your work.

The component commands allow you to do a single step at a time and begin with dbt-meshify operation. For example, if you wanted to add a new version to a model, you would run something like dbt-meshify operation add-version --select fct_orders. This command would:

  1. add version configuration values to fct_orders

and that's it!

The global commands combine multiple component commands to complete a larger set of work and begin with dbt-meshify. For example, if you wanted to define a group for a subset of your models, you would run something like dbt-meshify group finance --owner-name \"Monopoly Man\" --select +tag:finance. This command would:

  1. define a new group named \"finance\" in your dbt project, setting the owner name to \"Monopoly Man\"
  2. add all models tagged with \"finance\" to that new group
  3. set access to protected for all \"leaf\" models (models with no downstream dependencies) and models with cross-group dependencies
  4. add contracts to all protected nodes

all at once!

The next question to ask yourself is \"which of my models do I want to add these features to?\". This informs the selection syntax you provide to the dbt-meshify command of choice. dbt-meshify uses the same selection syntax as dbt, so you can use the --select, --exclude and --selector flags to select resources based on model names, tags, and so on!

Once you've decided:

  1. which feature(s) you want to add to your dbt project
  2. which subset of models you want to add those feature(s) to

you're ready to use dbt-meshify!

For further information, check out the available commands or read through some examples.

"},{"location":"#what-dbt-meshify-does-not-handle","title":"What dbt-meshify does not handle","text":"

There are a handful of known edge cases that this package does not automatically handle. In these cases, we recommend doing a manual check to make sure you've handled these appropriately:

edge case manual check dbt-meshify split copies over the entire contents of the packages.yml file from the original project to the new subproject remove unnecessary packages from each project dbt-meshify split makes a copy of all necessary macros from the original project to the new subproject consider creating a private \"macros only\" project to install as a package into all of your other projects, instead of maintaining duplicate copies of shared macros"},{"location":"commands/","title":"dbt-meshify Commands","text":""},{"location":"commands/#dbt-meshify","title":"dbt-meshify","text":"

Usage:

dbt-meshify [OPTIONS] COMMAND [ARGS]...\n

Options:

  --dry-run\n  --debug\n  --version  Show version information and exit\n  --help     Show this message and exit.\n

Subcommands

"},{"location":"commands/#dbt-meshify-connect","title":"dbt-meshify connect","text":"

Connects multiple dbt projects together by adding all necessary dbt Mesh constructs

Usage:

dbt-meshify connect [OPTIONS]\n

Options:

  --project-paths TUPLE         The paths to the set of dbt projects to\n                                connect. Must supply 2+ paths.\n  --projects-dir PATH           The path to a directory containing multiple\n                                dbt projects. Directory must contain 2+\n                                projects.\n  -e, --exclude-projects TUPLE  The set of dbt projects to exclude from the\n                                operation when using the --projects-dir\n                                option.\n  -r, --read-catalog            Skips the dbt docs generate step and reads the\n                                local catalog.json file.\n  --help                        Show this message and exit.\n
"},{"location":"commands/#dbt-meshify-group","title":"dbt-meshify group","text":"

Creates a new dbt group based on the selection syntax Detects the edges of the group, makes their access protected, and adds contracts to them

Usage:

dbt-meshify group [OPTIONS] NAME\n

Options:

  -e, --exclude TUPLE      The dbt selection syntax specifying the resources\n                           to exclude in the operation\n  --group-yml-path PATH    An optional path to store the new group YAML\n                           definition.\n  --owner-email TEXT       The group Owner's email address.\n  --owner-name TEXT        The group Owner's name.\n  --owner-properties TEXT  Additional properties to assign to a group Owner.\n  --project-path PATH      The path to the dbt project to operate on. Defaults\n                           to the current directory.\n  -r, --read-catalog       Skips the dbt docs generate step and reads the\n                           local catalog.json file.\n  -s, --select TUPLE       The dbt selection syntax specifying the resources\n                           to include in the operation\n  --selector TUPLE         The name(s) of the YML selector specifying the\n                           resources to include in the operation\n  --help                   Show this message and exit.\n
"},{"location":"commands/#dbt-meshify-operation","title":"dbt-meshify operation","text":"

Set of subcommands for performing mesh operations on dbt projects

Usage:

dbt-meshify operation [OPTIONS] COMMAND [ARGS]...\n

Options:

  --help  Show this message and exit.\n

Subcommands

"},{"location":"commands/#dbt-meshify-operation-add-contract","title":"dbt-meshify operation add-contract","text":"

Adds a contract to all selected models.

Usage:

dbt-meshify operation add-contract [OPTIONS]\n

Options:

  -e, --exclude TUPLE  The dbt selection syntax specifying the resources to\n                       exclude in the operation\n  --project-path PATH  The path to the dbt project to operate on. Defaults to\n                       the current directory.\n  -r, --read-catalog   Skips the dbt docs generate step and reads the local\n                       catalog.json file.\n  -s, --select TUPLE   The dbt selection syntax specifying the resources to\n                       include in the operation\n  --selector TUPLE     The name(s) of the YML selector specifying the\n                       resources to include in the operation\n  --help               Show this message and exit.\n
"},{"location":"commands/#dbt-meshify-operation-add-version","title":"dbt-meshify operation add-version","text":"

Adds model version boilerplate for all selected models.

Usage:

dbt-meshify operation add-version [OPTIONS]\n

Options:

  -e, --exclude TUPLE  The dbt selection syntax specifying the resources to\n                       exclude in the operation\n  --project-path PATH  The path to the dbt project to operate on. Defaults to\n                       the current directory.\n  -r, --read-catalog   Skips the dbt docs generate step and reads the local\n                       catalog.json file.\n  -s, --select TUPLE   The dbt selection syntax specifying the resources to\n                       include in the operation\n  --selector TUPLE     The name(s) of the YML selector specifying the\n                       resources to include in the operation\n  --defined-in TEXT\n  --help               Show this message and exit.\n
"},{"location":"commands/#dbt-meshify-operation-bump-version","title":"dbt-meshify operation bump-version","text":"

Create new model versions for all selected models.

Usage:

dbt-meshify operation bump-version [OPTIONS]\n

Options:

  -e, --exclude TUPLE  The dbt selection syntax specifying the resources to\n                       exclude in the operation\n  --project-path PATH  The path to the dbt project to operate on. Defaults to\n                       the current directory.\n  -r, --read-catalog   Skips the dbt docs generate step and reads the local\n                       catalog.json file.\n  -s, --select TUPLE   The dbt selection syntax specifying the resources to\n                       include in the operation\n  --selector TUPLE     The name(s) of the YML selector specifying the\n                       resources to include in the operation\n  -l, --latest         Makes the newest version the latest version when\n                       creating model versions\n  --increment, --i     Increments the latest_version setting by 1 when\n                       creating model versions\n  --defined-in TEXT\n  --help               Show this message and exit.\n
"},{"location":"commands/#dbt-meshify-operation-create-group","title":"dbt-meshify operation create-group","text":"

Create a group and add selected resources to the group.

Usage:

dbt-meshify operation create-group [OPTIONS] NAME\n

Options:

  -e, --exclude TUPLE      The dbt selection syntax specifying the resources\n                           to exclude in the operation\n  --group-yml-path PATH    An optional path to store the new group YAML\n                           definition.\n  --owner-email TEXT       The group Owner's email address.\n  --owner-name TEXT        The group Owner's name.\n  --owner-properties TEXT  Additional properties to assign to a group Owner.\n  --project-path PATH      The path to the dbt project to operate on. Defaults\n                           to the current directory.\n  -r, --read-catalog       Skips the dbt docs generate step and reads the\n                           local catalog.json file.\n  -s, --select TUPLE       The dbt selection syntax specifying the resources\n                           to include in the operation\n  --selector TUPLE         The name(s) of the YML selector specifying the\n                           resources to include in the operation\n  --help                   Show this message and exit.\n
"},{"location":"commands/#dbt-meshify-split","title":"dbt-meshify split","text":"

Splits out a new subproject from a dbt project by adding all necessary dbt Mesh constructs to the resources based on the selected resources.

Usage:

dbt-meshify split [OPTIONS] PROJECT_NAME\n

Options:

  --create-path PATH   The path to create the new dbt project. Defaults to the\n                       name argument supplied.\n  -e, --exclude TUPLE  The dbt selection syntax specifying the resources to\n                       exclude in the operation\n  --project-path PATH  The path to the dbt project to operate on. Defaults to\n                       the current directory.\n  -r, --read-catalog   Skips the dbt docs generate step and reads the local\n                       catalog.json file.\n  -s, --select TUPLE   The dbt selection syntax specifying the resources to\n                       include in the operation\n  --selector TUPLE     The name(s) of the YML selector specifying the\n                       resources to include in the operation\n  --help               Show this message and exit.\n
"},{"location":"commands/#dbt-meshify-version","title":"dbt-meshify version","text":"

Increment the models to the next version, and create in the initial version if it has not yet been defined.

Usage:

dbt-meshify version [OPTIONS]\n

Options:

  -e, --exclude TUPLE  The dbt selection syntax specifying the resources to\n                       exclude in the operation\n  --project-path PATH  The path to the dbt project to operate on. Defaults to\n                       the current directory.\n  -r, --read-catalog   Skips the dbt docs generate step and reads the local\n                       catalog.json file.\n  -s, --select TUPLE   The dbt selection syntax specifying the resources to\n                       include in the operation\n  --selector TUPLE     The name(s) of the YML selector specifying the\n                       resources to include in the operation\n  -l, --latest         Makes the newest version the latest version when\n                       creating model versions\n  --increment, --i     Increments the latest_version setting by 1 when\n                       creating model versions\n  --defined-in TEXT\n  --help               Show this message and exit.\n
"},{"location":"examples/","title":"Examples","text":"

For consistency and clarity of the following examples, we're going to use a simplified dbt project. In practice, the model governance and cross-project lineage features describe are most beneficial for large dbt projects that are struggling to scale.

We will give a basic example for each command, but to see the full list of additional flags you can add to a given command, check out the commands page.

Note

One helpful flag that you can add to all of the commands is --read-catalog, which will skip the dbt docs generate step and instead read the local catalog.json file - this will speed up the time it takes to run the dbt-meshify commands but relies on your local catalog.json file being up-to-date. Alternatively, you can configure this via the DBT_MESHIFY_READ_CATALOG environment variable.

Let's imagine a dbt project with the following models:

You can checkout the source code for this example here.

"},{"location":"examples/#component-commands","title":"Component commands","text":""},{"location":"examples/#create-a-new-group","title":"Create a new group","text":"

Let's say you want to create a new group for your sales analytics models.

You can run the following command:

dbt-meshify operation create-group sales_analytics --owner-name Ralphie --select +int_sales__unioned +int_returns__unioned transactions\n

This will create a new group named \"sales_analytics\" with the owner \"Ralphie\" and add all selected models to that group with the appropriate access configuration:

"},{"location":"examples/#addincrement-model-versions","title":"Add/increment model versions","text":"

Let's say you want to add a new version to the customers model, which is currently un-versioned. Versions can provide a smoother upgrade pathway when introducing breaking changes to models that have downstream dependencies.

You can run the following command:

dbt-meshify version --select customers\n

This will add a version to the customers model for your current version, and will add a new version for breaking change you wish to implement:

"},{"location":"examples/#add-contracts","title":"Add contract(s)","text":"

Let's say you want to add a new contract to the stores model, which is currently un-contracted.

You can run the following command:

dbt-meshify operation add-contract --select stores\n

This will add an enforced contract to the stores model:

"},{"location":"examples/#global-commands","title":"Global commands","text":""},{"location":"examples/#group-together-a-subset-of-models","title":"Group together a subset of models","text":"

Let's say you want to group together your sales analytics models - create a new group and add contracts to appropriate models simultaneously.

You can run the following command:

dbt-meshify group sales_analytics --owner-name Ralphie --select +int_sales__unioned +int_returns__unioned transactions\n

This will create a new group named \"sales_analytics\" with the owner \"Ralphie\", add all selected models to that group with the appropriate access configuration, _and add contracts to the models at the boundary between this group and the rest of the project__:

"},{"location":"examples/#split-out-a-new-subproject","title":"Split out a new subproject","text":"

Let's say you want to split our your sales analytics models into a new subproject.

You can run the following command:

dbt-meshify split sales_analytics --select +int_sales__unioned +int_returns__unioned transactions\n

This will create a new subproject that contains the selected sales analytics models, configure the \"edge\" models to be public and contracted, and replace all dependencies in the downstream project on the upstreams's models with cross-project refs:

By default, the new subproject will be created in the current directory; however, you can use the --create-path flag to create it in any directory you like.

"},{"location":"examples/#connect-multiple-dbt-projects","title":"Connect multiple dbt projects","text":"

Let's look at a slightly modified version of the example we've been working with. Instead of a single dbt project, let's imagine you're starting with two separate dbt projects connected via the \"source hack\":

We call this type of multi-project configuration the \"source hack\" because there are models generated by project A (stores and customers) that are defined as sources in project B.

Let's say we want to connect these two projects using model governance best practices and cross project refs.

You can run the following command:

dbt-meshify connect --project-paths path/to/project_a path/to/project_b\n

This will make the upstream project a dependency for the downstream project, configure the \"edge\" models to be public and contracted, and replace all dependencies in the downstream project on the upstreams's models with cross-project refs:

"}]} \ No newline at end of file diff --git a/0.5/sitemap.xml.gz b/0.5/sitemap.xml.gz index e832387..c7f98a4 100644 Binary files a/0.5/sitemap.xml.gz and b/0.5/sitemap.xml.gz differ