From bfbbc1f7a49357f1c4f63186b6854261bc926e12 Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Wed, 13 Nov 2024 19:50:26 -0500 Subject: [PATCH 1/2] Adding clarifying language (#6473) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## What are you changing in this pull request and why? See [internal chat](https://dbt-labs.slack.com/archives/C02NCQ9483C/p1731336895776299) for context ## Checklist - [ ] I have reviewed the [Content style guide](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/content-style-guide.md) so my content adheres to these guidelines. - [ ] The topic I'm writing about is for specific dbt version(s) and I have versioned it according to the [version a whole page](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#adding-a-new-version) and/or [version a block of content](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#versioning-blocks-of-content) guidelines. - [ ] I have added checklist item(s) to this list for anything anything that needs to happen before this PR is merged, such as "needs technical review" or "change base branch." - [ ] The content in this PR requires a dbt release note, so I added one to the [release notes page](https://docs.getdbt.com/docs/dbt-versions/dbt-cloud-release-notes). --- 🚀 Deployment available! Here are the direct links to the updated files: - https://docs-getdbt-com-git-snowflake-auth-dbt-labs.vercel.app/docs/cloud/connect-data-platform/connect-snowflake --------- Co-authored-by: Leona B. Campbell <3880403+runleonarun@users.noreply.github.com> --- .../docs/docs/cloud/connect-data-platform/connect-snowflake.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/cloud/connect-data-platform/connect-snowflake.md b/website/docs/docs/cloud/connect-data-platform/connect-snowflake.md index d8dd8dfec11..7e4bc7a9288 100644 --- a/website/docs/docs/cloud/connect-data-platform/connect-snowflake.md +++ b/website/docs/docs/cloud/connect-data-platform/connect-snowflake.md @@ -19,7 +19,7 @@ The following fields are required when creating a Snowflake connection ## Authentication methods -This section describes the different authentication methods available for connecting dbt Cloud to Snowflake. +This section describes the different authentication methods for connecting dbt Cloud to Snowflake. Configure Deployment environment (Production, Staging, General) credentials globally in the [**Connections**](/docs/deploy/deploy-environments#deployment-connection) area of **Account settings**. Individual users configure their development credentials in the [**Credentials**](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud#get-started-with-the-cloud-ide) area of their user profile. ### Username / Password From 988f1b7189b511943a006dc8d14a944f351d4b82 Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Wed, 13 Nov 2024 17:01:20 -0800 Subject: [PATCH 2/2] Adding multiple unique keys (#6438) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes https://github.com/dbt-labs/docs.getdbt.com/issues/6343 ## What are you changing in this pull request and why? Adds multiple unique keys as outline in [this plan](https://github.com/dbt-labs/docs.getdbt.com/issues/6343#issuecomment-2450966778). ## Checklist - [ ] I have reviewed the [Content style guide](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/content-style-guide.md) so my content adheres to these guidelines. - [ ] The topic I'm writing about is for specific dbt version(s) and I have versioned it according to the [version a whole page](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#adding-a-new-version) and/or [version a block of content](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#versioning-blocks-of-content) guidelines. - [ ] I have added checklist item(s) to this list for anything anything that needs to happen before this PR is merged, such as "needs technical review" or "change base branch." - [ ] The content in this PR requires a dbt release note, so I added one to the [release notes page](https://docs.getdbt.com/docs/dbt-versions/dbt-cloud-release-notes). --- 🚀 Deployment available! Here are the direct links to the updated files: - https://docs-getdbt-com-git-mult-unique-keys-dbt-labs.vercel.app/docs/build/snapshots - https://docs-getdbt-com-git-mult-unique-keys-dbt-labs.vercel.app/reference/resource-configs/unique_key --------- Co-authored-by: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> --- website/docs/docs/build/snapshots.md | 2 +- .../reference/resource-configs/unique_key.md | 47 +++++++------------ 2 files changed, 19 insertions(+), 30 deletions(-) diff --git a/website/docs/docs/build/snapshots.md b/website/docs/docs/build/snapshots.md index dd7a44fd48c..e5f33b11cab 100644 --- a/website/docs/docs/build/snapshots.md +++ b/website/docs/docs/build/snapshots.md @@ -83,7 +83,7 @@ The following table outlines the configurations available for snapshots: | [schema](/reference/resource-configs/schema) | Specify a custom schema for the snapshot | No | snapshots | | [alias](/reference/resource-configs/alias) | Specify an alias for the snapshot | No | your_custom_snapshot | | [strategy](/reference/resource-configs/strategy) | The snapshot strategy to use. Valid values: `timestamp` or `check` | Yes | timestamp | -| [unique_key](/reference/resource-configs/unique_key) | A column or expression for the record | Yes | id | +| [unique_key](/reference/resource-configs/unique_key) | A column(s) (string or array) or expression for the record | Yes | `id` or `[order_id, product_id]` | | [check_cols](/reference/resource-configs/check_cols) | If using the `check` strategy, then the columns to check | Only if using the `check` strategy | ["status"] | | [updated_at](/reference/resource-configs/updated_at) | If using the `timestamp` strategy, the timestamp column to compare | Only if using the `timestamp` strategy | updated_at | | [invalidate_hard_deletes](/reference/resource-configs/invalidate_hard_deletes) | Find hard deleted records in source and set `dbt_valid_to` to current time if the record no longer exists | No | True | diff --git a/website/docs/reference/resource-configs/unique_key.md b/website/docs/reference/resource-configs/unique_key.md index 996e7148292..41884e175d2 100644 --- a/website/docs/reference/resource-configs/unique_key.md +++ b/website/docs/reference/resource-configs/unique_key.md @@ -1,6 +1,6 @@ --- resource_types: [snapshots] -description: "Unique_key - Read this in-depth guide to learn about configurations in dbt." +description: "Learn more about unique_key configurations in dbt." datatype: column_name_or_expression --- @@ -14,7 +14,7 @@ snapshots: - name: orders_snapshot relation: source('my_source', 'my_table') [config](/reference/snapshot-configs): - unique_key: id + unique_key: order_id ``` @@ -52,7 +52,7 @@ snapshots: ## Description A column name or expression that is unique for the inputs of a snapshot. dbt uses this to match records between a result set and an existing snapshot, so that changes can be captured correctly. -In Versionless and dbt v1.9 and later, [snapshots](/docs/build/snapshots) are defined and configured in YAML files within your `snapshots/` directory. The `unique_key` is specified within the `config` block of your snapshot YAML file. +In Versionless and dbt v1.9 and later, [snapshots](/docs/build/snapshots) are defined and configured in YAML files within your `snapshots/` directory. You can specify one or multiple `unique_key` values within your snapshot YAML file's `config` key. :::caution @@ -114,29 +114,37 @@ snapshots: -### Use a combination of two columns as a unique key -This configuration accepts a valid column expression. As such, you can concatenate two columns together as a unique key if required. It's a good idea to use a separator (e.g. `'-'`) to ensure uniqueness. - +### Use multiple unique keys + +You can configure snapshots to use multiple unique keys for `primary_key` columns. + ```yaml snapshots: - - name: transaction_items_snapshot - relation: source('erp', 'transactions') + - name: orders_snapshot + relation: source('jaffle_shop', 'orders') config: schema: snapshots - unique_key: "transaction_id || '-' || line_item_id" + unique_key: + - order_id + - product_id strategy: timestamp updated_at: updated_at - + ``` + +### Use a combination of two columns as a unique key + +This configuration accepts a valid column expression. As such, you can concatenate two columns together as a unique key if required. It's a good idea to use a separator (for example, `'-'`) to ensure uniqueness. + ```jinja2 @@ -159,25 +167,9 @@ from {{ source('erp', 'transactions') }} ``` - Though, it's probably a better idea to construct this column in your query and use that as the `unique_key`: - - - - -```yaml -snapshots: - - name: transaction_items_snapshot - relation: {{ ref('transaction_items_ephemeral') }} - config: - schema: snapshots - unique_key: id - strategy: timestamp - updated_at: updated_at -``` - @@ -195,9 +187,6 @@ from {{ source('erp', 'transactions') }} In this example, we create an ephemeral model `transaction_items_ephemeral` that creates an `id` column that can be used as the `unique_key` our snapshot configuration. - - - ```jinja2