From cca64d35b1ec3cf592f84e3d77bcd7e6bcd07469 Mon Sep 17 00:00:00 2001 From: Ben Cassell <98852248+benc-db@users.noreply.github.com> Date: Tue, 17 Dec 2024 09:23:31 -0800 Subject: [PATCH 01/15] Update databricks-configs.md for merge settings --- website/docs/reference/resource-configs/databricks-configs.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/website/docs/reference/resource-configs/databricks-configs.md b/website/docs/reference/resource-configs/databricks-configs.md index 6ac3e23c113..53aa0fe4375 100644 --- a/website/docs/reference/resource-configs/databricks-configs.md +++ b/website/docs/reference/resource-configs/databricks-configs.md @@ -437,8 +437,8 @@ Beginning with 1.9, `merge` behavior can be modified with the following addition - `skip_not_matched_step`: If set to `true`, the 'not matched' clause will not be included. - `matched_condition`: Condition to apply to the `WHEN MATCHED` clause. You should use the `target_alias` and `source_alias` to write a conditional expression, such as `DBT_INTERNAL_DEST.col1 = hash(DBT_INTERNAL_SOURCE.col2, DBT_INTERNAL_SOURCE.col3)`. This condition further restricts the matched set of rows. - `not_matched_condition`: Condition to apply to the `WHEN NOT MATCHED [BY TARGET]` clause. This condition further restricts the set of rows in the target that do not match the source that will be inserted into the merged table. -- `not_matched_by_source_condition`: Condition to apply to the further filter `WHEN NOT MATCHED BY SOURCE` clause. Only used in conjunction with `not_matched_by_source_action: delete`. -- `not_matched_by_source_action`: If set to `delete`, a `DELETE` clause is added to the merge statement for `WHEN NOT MATCHED BY SOURCE`. +- `not_matched_by_source_condition`: Condition to apply to the further filter `WHEN NOT MATCHED BY SOURCE` clause. Only used in conjunction with `not_matched_by_source_action`. +- `not_matched_by_source_action`: Action to apply when the condition is met. For example: `update set t.attr1 = 'deleted', t.tech_change_ts = current_timestamp()`. - `merge_with_schema_evolution`: If set to `true`, the merge statement includes the `WITH SCHEMA EVOLUTION` clause. For more details on the meaning of each merge clause, please see [the Databricks documentation](https://docs.databricks.com/en/sql/language-manual/delta-merge-into.html). From 8f4bdbd6e78c3cbc235de8c78a7490e9bad5d006 Mon Sep 17 00:00:00 2001 From: Ben Cassell <98852248+benc-db@users.noreply.github.com> Date: Tue, 17 Dec 2024 10:03:39 -0800 Subject: [PATCH 02/15] Update databricks-configs.md --- website/docs/reference/resource-configs/databricks-configs.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/reference/resource-configs/databricks-configs.md b/website/docs/reference/resource-configs/databricks-configs.md index 53aa0fe4375..d7b5a79db37 100644 --- a/website/docs/reference/resource-configs/databricks-configs.md +++ b/website/docs/reference/resource-configs/databricks-configs.md @@ -438,7 +438,7 @@ Beginning with 1.9, `merge` behavior can be modified with the following addition - `matched_condition`: Condition to apply to the `WHEN MATCHED` clause. You should use the `target_alias` and `source_alias` to write a conditional expression, such as `DBT_INTERNAL_DEST.col1 = hash(DBT_INTERNAL_SOURCE.col2, DBT_INTERNAL_SOURCE.col3)`. This condition further restricts the matched set of rows. - `not_matched_condition`: Condition to apply to the `WHEN NOT MATCHED [BY TARGET]` clause. This condition further restricts the set of rows in the target that do not match the source that will be inserted into the merged table. - `not_matched_by_source_condition`: Condition to apply to the further filter `WHEN NOT MATCHED BY SOURCE` clause. Only used in conjunction with `not_matched_by_source_action`. -- `not_matched_by_source_action`: Action to apply when the condition is met. For example: `update set t.attr1 = 'deleted', t.tech_change_ts = current_timestamp()`. +- `not_matched_by_source_action`: Action to apply when the condition is met. For example: `not_matched_by_source_action: "update set t.attr1 = 'deleted', t.tech_change_ts = current_timestamp()"`. - `merge_with_schema_evolution`: If set to `true`, the merge statement includes the `WITH SCHEMA EVOLUTION` clause. For more details on the meaning of each merge clause, please see [the Databricks documentation](https://docs.databricks.com/en/sql/language-manual/delta-merge-into.html). From ce615e228ad7494e70ced88f5e2c18aabda70269 Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Wed, 18 Dec 2024 13:34:56 -0500 Subject: [PATCH 03/15] Update website/docs/reference/resource-configs/databricks-configs.md --- website/docs/reference/resource-configs/databricks-configs.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/reference/resource-configs/databricks-configs.md b/website/docs/reference/resource-configs/databricks-configs.md index d7b5a79db37..1ee89efc95c 100644 --- a/website/docs/reference/resource-configs/databricks-configs.md +++ b/website/docs/reference/resource-configs/databricks-configs.md @@ -438,7 +438,7 @@ Beginning with 1.9, `merge` behavior can be modified with the following addition - `matched_condition`: Condition to apply to the `WHEN MATCHED` clause. You should use the `target_alias` and `source_alias` to write a conditional expression, such as `DBT_INTERNAL_DEST.col1 = hash(DBT_INTERNAL_SOURCE.col2, DBT_INTERNAL_SOURCE.col3)`. This condition further restricts the matched set of rows. - `not_matched_condition`: Condition to apply to the `WHEN NOT MATCHED [BY TARGET]` clause. This condition further restricts the set of rows in the target that do not match the source that will be inserted into the merged table. - `not_matched_by_source_condition`: Condition to apply to the further filter `WHEN NOT MATCHED BY SOURCE` clause. Only used in conjunction with `not_matched_by_source_action`. -- `not_matched_by_source_action`: Action to apply when the condition is met. For example: `not_matched_by_source_action: "update set t.attr1 = 'deleted', t.tech_change_ts = current_timestamp()"`. +- `not_matched_by_source_action`: The action to apply when the condition is met. Configure as an expression. For example: `not_matched_by_source_action: "update set t.attr1 = 'deleted', t.tech_change_ts = current_timestamp()"`. - `merge_with_schema_evolution`: If set to `true`, the merge statement includes the `WITH SCHEMA EVOLUTION` clause. For more details on the meaning of each merge clause, please see [the Databricks documentation](https://docs.databricks.com/en/sql/language-manual/delta-merge-into.html). From 38013cc3dc25e592651d84634e54b5a3b2bbef3c Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Fri, 20 Dec 2024 16:59:58 +0000 Subject: [PATCH 04/15] remove description value from model notifications example this pr removes the `description` field/value from the groups.yml example. this is to make sure it isn't confused with the `description` config that dbt uses. pr also adds a callout clarifying that the `owner` field accepts any input (liek favorite_food). --- website/docs/docs/deploy/model-notifications.md | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/website/docs/docs/deploy/model-notifications.md b/website/docs/docs/deploy/model-notifications.md index 45ffbef0a4f..7396f67f14d 100644 --- a/website/docs/docs/deploy/model-notifications.md +++ b/website/docs/docs/deploy/model-notifications.md @@ -38,21 +38,29 @@ groups: - name: finance owner: # Email is required to receive model-level notifications, additional properties are also allowed. - name: "Finance Team" - description: "Models related to the finance department" + name: "Finance team" email: finance@dbtlabs.com favorite_food: donuts - name: marketing owner: - name: "Marketing Team" - description: "Models related to the marketing department" + name: "Marketing team" email: marketing@dbtlabs.com favorite_food: jaffles + + - name: docs + owner: + name: "Documentation team" + email: docs@dbtlabs.com + favorite_food: pizza ``` +:::tip +The `owner` value is flexible and accepts arbitrary inputs in addition to the required `email` field. For example, you could include a custom field like `favorite_food` to add context about the team. +::: + ## Attach groups to models Attach groups to models as you would any other config, in either the `dbt_project.yml` or `whatever.yml` files. For example: From cb4e20bb20f09c73f212fe54365bb461d4d60cdf Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Fri, 20 Dec 2024 17:00:44 +0000 Subject: [PATCH 05/15] Update website/docs/docs/deploy/model-notifications.md --- website/docs/docs/deploy/model-notifications.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/deploy/model-notifications.md b/website/docs/docs/deploy/model-notifications.md index 7396f67f14d..2a2dbc45e78 100644 --- a/website/docs/docs/deploy/model-notifications.md +++ b/website/docs/docs/deploy/model-notifications.md @@ -58,7 +58,7 @@ groups: :::tip -The `owner` value is flexible and accepts arbitrary inputs in addition to the required `email` field. For example, you could include a custom field like `favorite_food` to add context about the team. +The `owner` key is flexible and accepts arbitrary inputs in addition to the required `email` field. For example, you could include a custom field like `favorite_food` to add context about the team. ::: ## Attach groups to models From 7ff1078ff58f9cdbdc37ef1249fc1f02b7d79d9d Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Mon, 23 Dec 2024 12:28:15 +0000 Subject: [PATCH 06/15] replace callout wtih component --- website/docs/reference/resource-configs/batch_size.md | 2 +- website/docs/reference/resource-configs/begin.md | 2 +- website/docs/reference/resource-configs/delimiter.md | 2 +- website/docs/reference/resource-configs/event-time.md | 2 +- website/docs/reference/resource-configs/hard-deletes.md | 3 +-- website/docs/reference/resource-configs/lookback.md | 3 +-- .../reference/resource-properties/concurrent_batches.md | 6 +----- website/docs/reference/resource-properties/unit-tests.md | 5 +---- website/src/components/versionCallout/index.js | 2 +- 9 files changed, 9 insertions(+), 18 deletions(-) diff --git a/website/docs/reference/resource-configs/batch_size.md b/website/docs/reference/resource-configs/batch_size.md index 4001545778a..0110da53bb2 100644 --- a/website/docs/reference/resource-configs/batch_size.md +++ b/website/docs/reference/resource-configs/batch_size.md @@ -7,7 +7,7 @@ description: "dbt uses `batch_size` to determine how large batches are when runn datatype: hour | day | month | year --- -Available in the [dbt Cloud "Latest" release track](/docs/dbt-versions/cloud-release-tracks) and dbt Core v1.9 and higher. + ## Definition diff --git a/website/docs/reference/resource-configs/begin.md b/website/docs/reference/resource-configs/begin.md index dd47419be21..924cdfbcfad 100644 --- a/website/docs/reference/resource-configs/begin.md +++ b/website/docs/reference/resource-configs/begin.md @@ -7,7 +7,7 @@ description: "dbt uses `begin` to determine when a microbatch incremental model datatype: string --- -Available in the [dbt Cloud "Latest" release track](/docs/dbt-versions/cloud-release-tracks) and dbt Core v1.9 and higher. + ## Definition diff --git a/website/docs/reference/resource-configs/delimiter.md b/website/docs/reference/resource-configs/delimiter.md index 5cc5ddaf44b..e1a201ef198 100644 --- a/website/docs/reference/resource-configs/delimiter.md +++ b/website/docs/reference/resource-configs/delimiter.md @@ -4,7 +4,7 @@ datatype: default_value: "," --- -Supported in v1.7 and higher. + ## Definition diff --git a/website/docs/reference/resource-configs/event-time.md b/website/docs/reference/resource-configs/event-time.md index c18c8de6397..e746b7658ba 100644 --- a/website/docs/reference/resource-configs/event-time.md +++ b/website/docs/reference/resource-configs/event-time.md @@ -7,7 +7,7 @@ description: "dbt uses event_time to understand when an event occurred. When def datatype: string --- -Available in [the dbt Cloud "Latest" release track](/docs/dbt-versions/cloud-release-tracks) and dbt Core v1.9 and higher. + diff --git a/website/docs/reference/resource-configs/hard-deletes.md b/website/docs/reference/resource-configs/hard-deletes.md index 50c8046f4e1..859e4e9e31a 100644 --- a/website/docs/reference/resource-configs/hard-deletes.md +++ b/website/docs/reference/resource-configs/hard-deletes.md @@ -8,8 +8,7 @@ id: "hard-deletes" sidebar_label: "hard_deletes" --- -Available from dbt v1.9 or with [dbt Cloud "Latest" release track](/docs/dbt-versions/cloud-release-tracks). - + diff --git a/website/docs/reference/resource-configs/lookback.md b/website/docs/reference/resource-configs/lookback.md index 037ffdeb68f..fc832fab3d9 100644 --- a/website/docs/reference/resource-configs/lookback.md +++ b/website/docs/reference/resource-configs/lookback.md @@ -7,8 +7,7 @@ description: "dbt uses `lookback` to detrmine how many 'batches' of `batch_size` datatype: int --- -Available in the [dbt Cloud "Latest" release track](/docs/dbt-versions/cloud-release-tracks) and dbt Core v1.9 and higher. - + ## Definition Set the `lookback` to an integer greater than or equal to zero. The default value is `1`. You can configure `lookback` for a [model](/docs/build/models) in your `dbt_project.yml` file, property YAML file, or config block. diff --git a/website/docs/reference/resource-properties/concurrent_batches.md b/website/docs/reference/resource-properties/concurrent_batches.md index 4d6b2ea0af4..eef795344c3 100644 --- a/website/docs/reference/resource-properties/concurrent_batches.md +++ b/website/docs/reference/resource-properties/concurrent_batches.md @@ -5,11 +5,7 @@ datatype: model_name description: "Learn about concurrent_batches in dbt." --- -:::note - -Available in dbt Core v1.9+ or the [dbt Cloud "Latest" release tracks](/docs/dbt-versions/cloud-release-tracks). - -::: + diff --git a/website/docs/reference/resource-properties/unit-tests.md b/website/docs/reference/resource-properties/unit-tests.md index 7bc177a133c..46243f8c1ef 100644 --- a/website/docs/reference/resource-properties/unit-tests.md +++ b/website/docs/reference/resource-properties/unit-tests.md @@ -5,11 +5,8 @@ resource_types: [models] datatype: test --- -:::note + -This functionality is available in dbt Core v1.8+ and [dbt Cloud release tracks](/docs/dbt-versions/cloud-release-tracks). - -::: Unit tests validate your SQL modeling logic on a small set of static inputs before you materialize your full model in production. They support a test-driven development approach, improving both the efficiency of developers and reliability of code. diff --git a/website/src/components/versionCallout/index.js b/website/src/components/versionCallout/index.js index 598182c851f..975d6b6f1b9 100644 --- a/website/src/components/versionCallout/index.js +++ b/website/src/components/versionCallout/index.js @@ -13,7 +13,7 @@ const VersionCallout = ({ version }) => { Available from dbt v{version} or with the{' '} dbt Cloud "Latest" release track - {' '}. + {''}. From 26b86b2c5d48eb5245a0b2e50db78eeafd800447 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Mon, 23 Dec 2024 12:34:21 +0000 Subject: [PATCH 07/15] small fixes --- website/docs/docs/build/environment-variables.md | 1 - website/docs/reference/global-configs/version-compatibility.md | 2 +- 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/website/docs/docs/build/environment-variables.md b/website/docs/docs/build/environment-variables.md index 95242069ed9..4c0b785448e 100644 --- a/website/docs/docs/build/environment-variables.md +++ b/website/docs/docs/build/environment-variables.md @@ -105,7 +105,6 @@ dbt Cloud has a number of pre-defined variables built in. Variables are set auto The following environment variable is set automatically for the dbt Cloud IDE: - `DBT_CLOUD_GIT_BRANCH` — Provides the development Git branch name in the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud). - - Available in dbt v1.6 and later. - The variable changes when the branch is changed. - Doesn't require restarting the IDE after a branch change. - Currently not available in the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation). diff --git a/website/docs/reference/global-configs/version-compatibility.md b/website/docs/reference/global-configs/version-compatibility.md index 7667dcfda9c..b362add9842 100644 --- a/website/docs/reference/global-configs/version-compatibility.md +++ b/website/docs/reference/global-configs/version-compatibility.md @@ -14,7 +14,7 @@ Running with dbt=1.0.0 Found 13 models, 2 tests, 1 archives, 0 analyses, 204 macros, 2 operations.... ``` -:::info dbt Cloud release tracks +:::note dbt Cloud release tracks ::: From b303c101435594d4b1644d4bbe534d83ff523346 Mon Sep 17 00:00:00 2001 From: Natalie Fiann Date: Mon, 23 Dec 2024 14:33:41 +0000 Subject: [PATCH 08/15] Added global version callout --- website/docs/docs/build/unit-tests.md | 2 ++ .../reference/resource-configs/snapshot_meta_column_names.md | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/website/docs/docs/build/unit-tests.md b/website/docs/docs/build/unit-tests.md index fc4cf02b34f..a007f85d432 100644 --- a/website/docs/docs/build/unit-tests.md +++ b/website/docs/docs/build/unit-tests.md @@ -8,6 +8,8 @@ keywords: - unit test, unit tests, unit testing, dag --- + + :::note Unit testing functionality is available in [dbt Cloud Release Tracks](/docs/dbt-versions/cloud-release-tracks) or dbt Core v1.8+ diff --git a/website/docs/reference/resource-configs/snapshot_meta_column_names.md b/website/docs/reference/resource-configs/snapshot_meta_column_names.md index f1d29ba8bee..24e4c8ca577 100644 --- a/website/docs/reference/resource-configs/snapshot_meta_column_names.md +++ b/website/docs/reference/resource-configs/snapshot_meta_column_names.md @@ -6,7 +6,7 @@ default_value: {"dbt_valid_from": "dbt_valid_from", "dbt_valid_to": "dbt_valid_t id: "snapshot_meta_column_names" --- -Available in dbt Core v1.9+. Select v1.9 or newer from the version dropdown to view the configs. Try it now in the [dbt Cloud "Latest" release track](/docs/dbt-versions/cloud-release-tracks). + From 3ffe1586c7e3130e98b557172e1af39749ab95a9 Mon Sep 17 00:00:00 2001 From: nataliefiann <120089939+nataliefiann@users.noreply.github.com> Date: Mon, 23 Dec 2024 14:47:08 +0000 Subject: [PATCH 09/15] Update website/docs/docs/build/unit-tests.md --- website/docs/docs/build/unit-tests.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/unit-tests.md b/website/docs/docs/build/unit-tests.md index a007f85d432..b363aa524f8 100644 --- a/website/docs/docs/build/unit-tests.md +++ b/website/docs/docs/build/unit-tests.md @@ -8,7 +8,7 @@ keywords: - unit test, unit tests, unit testing, dag --- - + :::note From c6be034cd49c94640cf1c63c3bc7c8be78253a5c Mon Sep 17 00:00:00 2001 From: nataliefiann <120089939+nataliefiann@users.noreply.github.com> Date: Mon, 23 Dec 2024 14:49:26 +0000 Subject: [PATCH 10/15] Update website/docs/reference/resource-configs/snapshot_meta_column_names.md --- .../reference/resource-configs/snapshot_meta_column_names.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/reference/resource-configs/snapshot_meta_column_names.md b/website/docs/reference/resource-configs/snapshot_meta_column_names.md index 24e4c8ca577..57a1f3df1cc 100644 --- a/website/docs/reference/resource-configs/snapshot_meta_column_names.md +++ b/website/docs/reference/resource-configs/snapshot_meta_column_names.md @@ -6,7 +6,7 @@ default_value: {"dbt_valid_from": "dbt_valid_from", "dbt_valid_to": "dbt_valid_t id: "snapshot_meta_column_names" --- - + From 923a627b6c5f8f3c565861f6b12a0c400fc406ed Mon Sep 17 00:00:00 2001 From: nataliefiann <120089939+nataliefiann@users.noreply.github.com> Date: Mon, 23 Dec 2024 14:57:53 +0000 Subject: [PATCH 11/15] Update website/docs/reference/resource-configs/snapshot_meta_column_names.md --- .../reference/resource-configs/snapshot_meta_column_names.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/reference/resource-configs/snapshot_meta_column_names.md b/website/docs/reference/resource-configs/snapshot_meta_column_names.md index 57a1f3df1cc..24e4c8ca577 100644 --- a/website/docs/reference/resource-configs/snapshot_meta_column_names.md +++ b/website/docs/reference/resource-configs/snapshot_meta_column_names.md @@ -6,7 +6,7 @@ default_value: {"dbt_valid_from": "dbt_valid_from", "dbt_valid_to": "dbt_valid_t id: "snapshot_meta_column_names" --- - + From 172b532dbd90a0e877c51276ad9b796cdbe533e6 Mon Sep 17 00:00:00 2001 From: nataliefiann <120089939+nataliefiann@users.noreply.github.com> Date: Mon, 23 Dec 2024 15:15:22 +0000 Subject: [PATCH 12/15] Update website/docs/docs/build/unit-tests.md --- website/docs/docs/build/unit-tests.md | 1 - 1 file changed, 1 deletion(-) diff --git a/website/docs/docs/build/unit-tests.md b/website/docs/docs/build/unit-tests.md index b363aa524f8..5c8e0144ef5 100644 --- a/website/docs/docs/build/unit-tests.md +++ b/website/docs/docs/build/unit-tests.md @@ -10,7 +10,6 @@ keywords: -:::note Unit testing functionality is available in [dbt Cloud Release Tracks](/docs/dbt-versions/cloud-release-tracks) or dbt Core v1.8+ From 05abb568903df15019964662950f90643a37436e Mon Sep 17 00:00:00 2001 From: nataliefiann <120089939+nataliefiann@users.noreply.github.com> Date: Mon, 23 Dec 2024 15:15:40 +0000 Subject: [PATCH 13/15] Update website/docs/docs/build/unit-tests.md --- website/docs/docs/build/unit-tests.md | 1 - 1 file changed, 1 deletion(-) diff --git a/website/docs/docs/build/unit-tests.md b/website/docs/docs/build/unit-tests.md index 5c8e0144ef5..7604be6a1d8 100644 --- a/website/docs/docs/build/unit-tests.md +++ b/website/docs/docs/build/unit-tests.md @@ -11,7 +11,6 @@ keywords: -Unit testing functionality is available in [dbt Cloud Release Tracks](/docs/dbt-versions/cloud-release-tracks) or dbt Core v1.8+ ::: From a874248f21d2cc7c6ce9d289da8d21f98f86cc40 Mon Sep 17 00:00:00 2001 From: nataliefiann <120089939+nataliefiann@users.noreply.github.com> Date: Mon, 23 Dec 2024 15:15:56 +0000 Subject: [PATCH 14/15] Update website/docs/docs/build/unit-tests.md --- website/docs/docs/build/unit-tests.md | 1 - 1 file changed, 1 deletion(-) diff --git a/website/docs/docs/build/unit-tests.md b/website/docs/docs/build/unit-tests.md index 7604be6a1d8..a85ffa07ed2 100644 --- a/website/docs/docs/build/unit-tests.md +++ b/website/docs/docs/build/unit-tests.md @@ -12,7 +12,6 @@ keywords: -::: Historically, dbt's test coverage was confined to [โ€œdataโ€ tests](/docs/build/data-tests), assessing the quality of input data or resulting datasets' structure. However, these tests could only be executed _after_ building a model. From b214a2411727e921fe512c57f93461d1d6b62a37 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Mon, 23 Dec 2024 22:05:57 +0000 Subject: [PATCH 15/15] clarify full-refresh behavior (#6700) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit this pr clarifies full_refresh config behavior when you set it to `false` and `true`. Resolves #6698 --- ๐Ÿš€ Deployment available! Here are the direct links to the updated files: - https://docs-getdbt-com-git-update-full-refresh-dbt-labs.vercel.app/docs/build/incremental-microbatch - https://docs-getdbt-com-git-update-full-refresh-dbt-labs.vercel.app/reference/resource-configs/full_refresh --------- Co-authored-by: Leona B. Campbell <3880403+runleonarun@users.noreply.github.com> --- .../docs/docs/build/incremental-microbatch.md | 2 +- .../resource-configs/full_refresh.md | 35 +++++++++++-------- 2 files changed, 21 insertions(+), 16 deletions(-) diff --git a/website/docs/docs/build/incremental-microbatch.md b/website/docs/docs/build/incremental-microbatch.md index 4aff8b5839c..77f9ac274b0 100644 --- a/website/docs/docs/build/incremental-microbatch.md +++ b/website/docs/docs/build/incremental-microbatch.md @@ -232,7 +232,7 @@ from {{ source('sales', 'transactions') }} ### Full refresh -As a best practice, we recommend configuring `full_refresh: False` on microbatch models so that they ignore invocations with the `--full-refresh` flag. If you need to reprocess historical data, do so with a targeted backfill that specifies explicit start and end dates. +As a best practice, we recommend [configuring `full_refresh: false`](/reference/resource-configs/full_refresh) on microbatch models so that they ignore invocations with the `--full-refresh` flag. If you need to reprocess historical data, do so with a targeted backfill that specifies explicit start and end dates. ## Usage diff --git a/website/docs/reference/resource-configs/full_refresh.md b/website/docs/reference/resource-configs/full_refresh.md index c874fe7a396..5e291fa2454 100644 --- a/website/docs/reference/resource-configs/full_refresh.md +++ b/website/docs/reference/resource-configs/full_refresh.md @@ -4,6 +4,8 @@ description: "Setting the full_refresh config to false prevents a model or seed datatype: boolean --- +The `full_refresh` config allows you to control whether a resource will always or never perform a full-refresh. This config overrides the `--full-refresh` command-line flag. + ](/reference/resource-configs/resource-path): - +full_refresh: false - + +full_refresh: false | true ``` @@ -30,17 +31,14 @@ models: ```sql {{ config( - full_refresh = false + full_refresh = false | true ) }} select ... - ``` -The configured model(s) will not full-refresh when `dbt run --full-refresh` is invoked. - @@ -50,27 +48,34 @@ The configured model(s) will not full-refresh when `dbt run --full-refresh` is i ```yml seeds: [](/reference/resource-configs/resource-path): - +full_refresh: false + +full_refresh: false | true ``` -The configured seed(s) will not full-refresh when `dbt seed --full-refresh` is invoked. - +- If `full_refresh:true` — the configured resources(s) will full-refresh when `dbt run --full-refresh` is invoked. +- If `full_refresh:false` — the configured resources(s) will _not_ full-refresh when `dbt run --full-refresh` is invoked. + + ## Description -Optionally set a resource to always or never full-refresh. -- If specified as `true` or `false`, the -`full_refresh` config will take precedence over the presence or absence of the `--full-refresh` flag. -- If the `full_refresh` config is `none` or omitted, the resource will use the value of the `--full-refresh` flag. -**Note:** The `--full-refresh` flag also supports a short name, `-f`. +The `full_refresh` config allows you to optionally configure whether a resource will always or never perform a full-refresh. This config is an override for the `--full-refresh` command line flag used when running dbt commands. + + +| `full_refresh` value | Behavior | +| ---------------------------- | -------- | +| `true` | The resource always full-refreshes, regardless of the presence or absence of the `--full-refresh` flag. | +| `false` | The resource never full-refreshes, even if the `--full-refresh` flag is provided. | +| `none` or omitted | The resource follows the behavior of the `--full-refresh` flag. If the flag is used, the resource will full-refresh; otherwise, it won't. | -This logic is encoded in the [`should_full_refresh()`](https://github.com/dbt-labs/dbt-adapters/blob/60005a0a2bd33b61cb65a591bc1604b1b3fd25d5/dbt/include/global_project/macros/materializations/configs.sql) macro. +#### Note +- The `--full-refresh` flag also supports a short name, `-f`. +- The [`should_full_refresh()`](https://github.com/dbt-labs/dbt-adapters/blob/60005a0a2bd33b61cb65a591bc1604b1b3fd25d5/dbt/include/global_project/macros/materializations/configs.sql) macro has logic encoded. ## Usage