From eb93839faec4ef5fad5fa113d0f38d87e635fbb3 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Wed, 6 Nov 2024 11:54:57 +0000 Subject: [PATCH 01/13] add tags to exports --- website/docs/docs/build/saved-queries.md | 2 + .../docs/docs/dbt-versions/release-notes.md | 8 ++++ .../docs/use-dbt-semantic-layer/exports.md | 10 +++-- .../docs/reference/resource-configs/tags.md | 37 ++++++++++++++++++- 4 files changed, 53 insertions(+), 4 deletions(-) diff --git a/website/docs/docs/build/saved-queries.md b/website/docs/docs/build/saved-queries.md index ed56d13dcc9..d1fe785f483 100644 --- a/website/docs/docs/build/saved-queries.md +++ b/website/docs/docs/build/saved-queries.md @@ -58,6 +58,7 @@ Note that we use the double colon (::) to indicate whether a parameter is nested | `exports::config::export_as` | String | Required | Specifies the type of export: table, view, or upcoming cache options. Nested within `exports` and `config`. | | `exports::config::schema` | String | Optional | Schema for creating the table or view, not applicable for caching. Nested within `exports` and `config`. | | `exports::config::alias` | String | Optional | Table alias used to write to the table or view. This option can't be used for caching. Nested within `exports` and `config`. | +| `exports::config::tag` | String | Optional | Apply a [tag](/reference/resource-configs/tags) (or list of tags) to a resource. Tags help organize and filter resources in dbt. Nested within `exports` and `config`. | @@ -99,6 +100,7 @@ saved_queries: alias: my_export_alias export_as: table schema: my_export_schema_name + [tags](/reference/resource-configs/tags): ['my_tag'] ``` diff --git a/website/docs/docs/dbt-versions/release-notes.md b/website/docs/docs/dbt-versions/release-notes.md index c3f0bbfbe06..54326474e7c 100644 --- a/website/docs/docs/dbt-versions/release-notes.md +++ b/website/docs/docs/dbt-versions/release-notes.md @@ -24,6 +24,14 @@ Release notes are grouped by month for both multi-tenant and virtual private clo - Improved handling of queries when multiple tables are selected in a data source. - Fixed a bug when an IN filter contained a lot of values. - Better error messaging for queries that can't be parsed correctly. +- **New**: Exports now support [tags](/docs/reference/resource-configs/tags) in dbt Cloud. Tags allow you to categorize your resources and filter them in dbt Cloud. You can add tags to your [exports](/docs/build/saved-queries#configure-exports) in the `semantic_model.yml` file or `dbt_project.yml` file. For example: + + ```yml + exports: + - name: export_name + tags: ['export_tag'] + ... + ``` ## October 2024 diff --git a/website/docs/docs/use-dbt-semantic-layer/exports.md b/website/docs/docs/use-dbt-semantic-layer/exports.md index 5d6e4c0d996..af8e0dce1cd 100644 --- a/website/docs/docs/use-dbt-semantic-layer/exports.md +++ b/website/docs/docs/use-dbt-semantic-layer/exports.md @@ -29,16 +29,20 @@ The following section explains the main benefits of using exports, including: - [Easier changes](#easier-changes) - [Caching](#caching) -#### DRY representation + Currently, creating tables often involves generating tens, hundreds, or even thousands of tables that denormalize data into summary or metric mart tables. The main benefit of exports is creating a "Don't Repeat Yourself (DRY)" representation of the logic to construct each metric, dimension, join, filter, and so on. This allows you to reuse those components for long-term scalability, even if you're replacing manually written SQL models with references to the metrics or dimensions in saved queries. + -#### Easier changes + Exports ensure that changes to metrics and dimensions are made in one place and then cascade to those various destinations seamlessly. This prevents the problem of needing to update a metric across every model that references that same concept. + + + -#### Caching Use exports to pre-populate the cache, so that you're pre-computing what you need to serve users through the dynamic Semantic Layer APIs. + #### Considerations diff --git a/website/docs/reference/resource-configs/tags.md b/website/docs/reference/resource-configs/tags.md index f6c46f8a088..75d5c55a1af 100644 --- a/website/docs/reference/resource-configs/tags.md +++ b/website/docs/reference/resource-configs/tags.md @@ -30,6 +30,10 @@ seeds: [](/reference/resource-configs/resource-path): +tags: | [] +[exports](/docs/use-dbt-semantic-layer/exports): + [](/reference/resource-configs/resource-path): + +tags: | [] + ``` @@ -164,6 +168,32 @@ seeds: +### Apply tags to exports + + + +```yml +[exports](/docs/use-dbt-semantic-layer/exports): + jaffle_shop: + customer_order_metrics: + +tags: order_metrics +``` + + + + + +```yml +[exports](/docs/use-dbt-semantic-layer/exports): + jaffle_shop: + customer_order_metrics: + +tags: + - order_metrics + - hourly +``` + + + ## Usage notes ### Tags are additive @@ -178,7 +208,7 @@ Tags accumulate hierarchically. The above example would result in: ### Other resource types -Tags can also be applied to sources, exposures, and even _specific columns_ in a resource. +Tags can also be applied to [sources](/docs/build/sources), [exposures](/docs/build/exposures), [exports](/docs/use-dbt-semantic-layer/exports), and even _specific columns_ in a resource. These resources do not yet support the `config` property, so you'll need to specify the tags as a top-level key instead. @@ -206,6 +236,11 @@ sources: tests: - unique: tags: ['test_level'] + +exports: + - name: export_name + tags: ['export_tag'] + ... ``` From 744352134d1761094eb7e68a1f6abe34ba76c4af Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Wed, 6 Nov 2024 12:02:33 +0000 Subject: [PATCH 02/13] fix link --- website/docs/docs/build/saved-queries.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/saved-queries.md b/website/docs/docs/build/saved-queries.md index d1fe785f483..db978ece436 100644 --- a/website/docs/docs/build/saved-queries.md +++ b/website/docs/docs/build/saved-queries.md @@ -100,7 +100,7 @@ saved_queries: alias: my_export_alias export_as: table schema: my_export_schema_name - [tags](/reference/resource-configs/tags): ['my_tag'] + [tags](/reference/resource-configs/tags): 'my_tag' ``` From 44b92484699a3e07b41634e43f6fef16a2db84b0 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Wed, 6 Nov 2024 12:02:57 +0000 Subject: [PATCH 03/13] fix link --- website/docs/docs/dbt-versions/release-notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-versions/release-notes.md b/website/docs/docs/dbt-versions/release-notes.md index 54326474e7c..f7faac30645 100644 --- a/website/docs/docs/dbt-versions/release-notes.md +++ b/website/docs/docs/dbt-versions/release-notes.md @@ -24,7 +24,7 @@ Release notes are grouped by month for both multi-tenant and virtual private clo - Improved handling of queries when multiple tables are selected in a data source. - Fixed a bug when an IN filter contained a lot of values. - Better error messaging for queries that can't be parsed correctly. -- **New**: Exports now support [tags](/docs/reference/resource-configs/tags) in dbt Cloud. Tags allow you to categorize your resources and filter them in dbt Cloud. You can add tags to your [exports](/docs/build/saved-queries#configure-exports) in the `semantic_model.yml` file or `dbt_project.yml` file. For example: +- **New**: Exports now support [tags](/reference/resource-configs/tags) in dbt Cloud. Tags allow you to categorize your resources and filter them in dbt Cloud. You can add tags to your [exports](/docs/build/saved-queries#configure-exports) in the `semantic_model.yml` file or `dbt_project.yml` file. For example: ```yml exports: From 53c0351a95a72f6212768191b24ede084e48717b Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Wed, 6 Nov 2024 12:51:35 +0000 Subject: [PATCH 04/13] update example --- website/docs/docs/build/saved-queries.md | 2 +- .../docs/reference/resource-configs/tags.md | 28 +++++++++++++------ 2 files changed, 20 insertions(+), 10 deletions(-) diff --git a/website/docs/docs/build/saved-queries.md b/website/docs/docs/build/saved-queries.md index db978ece436..136474da3ee 100644 --- a/website/docs/docs/build/saved-queries.md +++ b/website/docs/docs/build/saved-queries.md @@ -58,7 +58,7 @@ Note that we use the double colon (::) to indicate whether a parameter is nested | `exports::config::export_as` | String | Required | Specifies the type of export: table, view, or upcoming cache options. Nested within `exports` and `config`. | | `exports::config::schema` | String | Optional | Schema for creating the table or view, not applicable for caching. Nested within `exports` and `config`. | | `exports::config::alias` | String | Optional | Table alias used to write to the table or view. This option can't be used for caching. Nested within `exports` and `config`. | -| `exports::config::tag` | String | Optional | Apply a [tag](/reference/resource-configs/tags) (or list of tags) to a resource. Tags help organize and filter resources in dbt. Nested within `exports` and `config`. | +| `exports::config::tags` | String | Optional | Apply a [tag](/reference/resource-configs/tags) (or list of tags) to a resource. Tags help organize and filter resources in dbt. Nested within `exports` and `config`. | diff --git a/website/docs/reference/resource-configs/tags.md b/website/docs/reference/resource-configs/tags.md index 75d5c55a1af..6018c774282 100644 --- a/website/docs/reference/resource-configs/tags.md +++ b/website/docs/reference/resource-configs/tags.md @@ -170,10 +170,12 @@ seeds: ### Apply tags to exports +This following example shows how to apply tags to an export in the `dbt_project.yml` file. The export is then tagged with `order_metrics`. + ```yml -[exports](/docs/use-dbt-semantic-layer/exports): +[exports](/docs/build/saved-queries-configure-exports): jaffle_shop: customer_order_metrics: +tags: order_metrics @@ -181,15 +183,23 @@ seeds: - +The second example shows how to apply tags to an export in the `semantic_model.yml` file. The export is then tagged with `order_metrics` and `hourly`. -```yml -[exports](/docs/use-dbt-semantic-layer/exports): - jaffle_shop: - customer_order_metrics: - +tags: - - order_metrics - - hourly + + +```yaml +saved_queries: + - name: order_metrics + ... + exports: + - name: hourly_order_metrics + config: + alias: my_export_alias + export_as: table + schema: my_export_schema_name + tags: + - order_metrics + - hourly ``` From c9a08209e458134e607a19811527bcf43a475eab Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Wed, 6 Nov 2024 12:58:11 +0000 Subject: [PATCH 05/13] fix link --- website/docs/reference/resource-configs/tags.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/website/docs/reference/resource-configs/tags.md b/website/docs/reference/resource-configs/tags.md index 6018c774282..992cc1ea22a 100644 --- a/website/docs/reference/resource-configs/tags.md +++ b/website/docs/reference/resource-configs/tags.md @@ -30,7 +30,7 @@ seeds: [](/reference/resource-configs/resource-path): +tags: | [] -[exports](/docs/use-dbt-semantic-layer/exports): +[exports](/docs/build/saved-queries#configure-exports): [](/reference/resource-configs/resource-path): +tags: | [] @@ -175,7 +175,7 @@ This following example shows how to apply tags to an export in the `dbt_project. ```yml -[exports](/docs/build/saved-queries-configure-exports): +[exports](/docs/build/saved-queries#configure-exports): jaffle_shop: customer_order_metrics: +tags: order_metrics From 8162530983af7e1be9457e7743d8c02993294ed1 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Mon, 18 Nov 2024 09:40:23 -0500 Subject: [PATCH 06/13] update tags --- website/docs/docs/build/saved-queries.md | 116 +++++++++++++++--- .../docs/reference/resource-configs/tags.md | 44 +++++-- 2 files changed, 138 insertions(+), 22 deletions(-) diff --git a/website/docs/docs/build/saved-queries.md b/website/docs/docs/build/saved-queries.md index 136474da3ee..68876b65160 100644 --- a/website/docs/docs/build/saved-queries.md +++ b/website/docs/docs/build/saved-queries.md @@ -17,8 +17,33 @@ To create a saved query, refer to the following table parameters. :::tip Note that we use the double colon (::) to indicate whether a parameter is nested within another parameter. So for example, `query_params::metrics` means the `metrics` parameter is nested under `query_params`. ::: + + + + +| Parameter | Type | Required | Description | +|-------|---------|----------|----------------| +| `name` | String | Required | Name of the saved query object. | +| `description` | String | Required | A description of the saved query. | +| `label` | String | Required | The display name for your saved query. This value will be shown in downstream tools. | +| `config` | String | Optional | Use the [`config`](/reference/resource-properties/config) property to specify configurations for your saved query. Supports `cache`, [`enabled`](/reference/resource-configs/enabled), `export_as`, [`group`](/reference/resource-configs/group), [`meta`](/reference/resource-configs/meta), and [`schema`](/reference/resource-configs/schema) configurations. | +| `config::cache::enabled` | Object | Optional | An object with a sub-key used to specify if a saved query should populate the [cache](/docs/use-dbt-semantic-layer/sl-cache). Accepts sub-key `true` or `false`. Defaults to `false` | +| `query_params` | Structure | Required | Contains the query parameters. | +| `query_params::metrics` | List or String | Optional | A list of the metrics to be used in the query as specified in the command line interface. | +| `query_params::group_by` | List or String | Optional | A list of the Entities and Dimensions to be used in the query, which include the `Dimension` or `TimeDimension`. | +| `query_params::where` | List or String | Optional | A list of strings that may include the `Dimension` or `TimeDimension` objects. | +| `exports` | List or Structure | Optional | A list of exports to be specified within the exports structure. | +| `exports::name` | String | Required | Name of the export object. | +| `exports::config` | List or Structure | Required | A [`config`](/reference/resource-properties/config) property for any parameters specifying the export. | +| `exports::config::export_as` | String | Required | The type of export to run. Options include table or view currently and cache in the near future. | +| `exports::config::schema` | String | Optional | The [schema](/reference/resource-configs/schema) for creating the table or view. This option cannot be used for caching. | +| `exports::config::alias` | String | Optional | The table [alias](/reference/resource-configs/alias) used to write to the table or view. This option cannot be used for caching. | +| `exports::config::tags` | String | Optional | Apply a [tag](/reference/resource-configs/tags) (or list of tags) to a resource. Tags help organize and filter resources in dbt. Nested within `exports` and `config`. | + + + - + | Parameter | Type | Required | Description | |-------|---------|----------|----------------| @@ -33,15 +58,15 @@ Note that we use the double colon (::) to indicate whether a parameter is nested | `query_params::where` | List or String | Optional | A list of strings that may include the `Dimension` or `TimeDimension` objects. | | `exports` | List or Structure | Optional | A list of exports to be specified within the exports structure. | | `exports::name` | String | Required | Name of the export object. | -| `exports::config` | List or Structure | Required | A config section for any parameters specifying the export. | +| `exports::config` | List or Structure | Required | A [`config`](/reference/resource-properties/config) property for any parameters specifying the export. | | `exports::config::export_as` | String | Required | The type of export to run. Options include table or view currently and cache in the near future. | | `exports::config::schema` | String | Optional | The schema for creating the table or view. This option cannot be used for caching. | -| `exports::config::alias` | String | Optional | The table alias used to write to the table or view. This option cannot be used for caching. | +| `exports::config::alias` | String | Optional | The table alias used to write to the table or view. This option cannot be used for caching. | - + | Parameter | Type | Required | Description | |-------|---------|----------|----------------| @@ -54,11 +79,10 @@ Note that we use the double colon (::) to indicate whether a parameter is nested | `query_params::where` | List or String | Optional | Conditions nested with the `query_params`: a list of strings that may include the `Dimension` or `TimeDimension` objects. | | `exports` | List or Structure | Optional | A list of exports to be specified within the exports structure. | | `exports::name` | String | Required | Name of export object, nested within `exports`. | -| `exports::config` | List or Structure | Required | A config section for any parameters specifying the export, nested within `exports`. | +| `exports::config` | List or Structure | Required | A [`config`](/reference/resource-properties/config) property for any parameters specifying the export, nested within `exports`. | | `exports::config::export_as` | String | Required | Specifies the type of export: table, view, or upcoming cache options. Nested within `exports` and `config`. | | `exports::config::schema` | String | Optional | Schema for creating the table or view, not applicable for caching. Nested within `exports` and `config`. | | `exports::config::alias` | String | Optional | Table alias used to write to the table or view. This option can't be used for caching. Nested within `exports` and `config`. | -| `exports::config::tags` | String | Optional | Apply a [tag](/reference/resource-configs/tags) (or list of tags) to a resource. Tags help organize and filter resources in dbt. Nested within `exports` and `config`. | @@ -70,13 +94,12 @@ Use saved queries to define and manage common Semantic Layer queries in YAML, in In your saved query config, you can also leverage [caching](/docs/use-dbt-semantic-layer/sl-cache) with the dbt Cloud job scheduler to cache common queries, speed up performance, and reduce compute costs. - - - + In the following example, you can set the saved query in the `semantic_model.yml` file: + ```yaml saved_queries: @@ -97,13 +120,45 @@ saved_queries: exports: - name: my_export config: + export_as: table alias: my_export_alias + schema: my_export_schema_name + tags: 'my_tag' +``` + + + + + +```yaml +saved_queries: + - name: test_saved_query + description: "{{ doc('saved_query_description') }}" + label: Test saved query + config: + cache: + enabled: true # Or false if you want it disabled by default + query_params: + metrics: + - simple_metric + group_by: + - "Dimension('user__ds')" + where: + - "{{ Dimension('user__ds', 'DAY') }} <= now()" + - "{{ Dimension('user__ds', 'DAY') }} >= '2023-01-01'" + exports: + - name: my_export + config: export_as: table + alias: my_export_alias schema: my_export_schema_name - [tags](/reference/resource-configs/tags): 'my_tag' ``` + + + + Note, that you can set `export_as` to both the saved query and the exports [config](/reference/resource-properties/config), with the exports config value taking precedence. If a key isn't set in the exports config, it will inherit the saved query config value. #### Where clause @@ -123,7 +178,6 @@ filter: | filter: | {{ Metric('metric_name', group_by=['entity_name']) }} ``` - @@ -149,8 +203,8 @@ saved_queries: exports: - name: my_export config: - alias: my_export_alias export_as: table + alias: my_export_alias schema: my_export_schema_name ``` @@ -183,6 +237,38 @@ Once you've configured your saved query and set the foundation block, you can no The following is an example of a saved query with an export: + + +```yaml +saved_queries: + - name: order_metrics + description: Relevant order metrics + query_params: + metrics: + - orders + - large_order + - food_orders + - order_total + group_by: + - Entity('order_id') + - TimeDimension('metric_time', 'day') + - Dimension('customer__customer_name') + - ... # Additional group_by + where: + - "{{TimeDimension('metric_time')}} > current_timestamp - interval '1 week'" + - ... # Additional where clauses + exports: + - name: order_metrics + config: + export_as: table # Options available: table, view + [alias](/reference/resource-configs/alias): my_export_alias # Optional - defaults to Export name + [schema](/reference/resource-configs/schema): my_export_schema_name # Optional - defaults to deployment schema + [tags](/reference/resource-configs/tags): 'my_tag' + +``` + + + ```yaml saved_queries: @@ -206,9 +292,11 @@ saved_queries: - name: order_metrics config: export_as: table # Options available: table, view - schema: YOUR_SCHEMA # Optional - defaults to deployment schema - alias: SOME_TABLE_NAME # Optional - defaults to Export name + schema: my_export_schema_name # Optional - defaults to deployment schema + alias: my_export_alias # Optional - defaults to Export name ``` + + ## Run exports diff --git a/website/docs/reference/resource-configs/tags.md b/website/docs/reference/resource-configs/tags.md index 992cc1ea22a..e8838a72422 100644 --- a/website/docs/reference/resource-configs/tags.md +++ b/website/docs/reference/resource-configs/tags.md @@ -16,6 +16,8 @@ datatype: string | [string] + + ```yml models: @@ -30,11 +32,32 @@ seeds: [](/reference/resource-configs/resource-path): +tags: | [] -[exports](/docs/build/saved-queries#configure-exports): +``` + + + + +```yml + +models: + [](/reference/resource-configs/resource-path): + +tags: | [] + +snapshots: + [](/reference/resource-configs/resource-path): + +tags: | [] + +seeds: + [](/reference/resource-configs/resource-path): + +tags: | [] + +exports: [](/reference/resource-configs/resource-path): +tags: | [] ``` + + @@ -170,6 +193,14 @@ seeds: ### Apply tags to exports + + +Applying tags to exports is only available in dbt Core versions 1.9 and later. + + + + + This following example shows how to apply tags to an export in the `dbt_project.yml` file. The export is then tagged with `order_metrics`. @@ -203,11 +234,12 @@ saved_queries: ``` + ## Usage notes ### Tags are additive -Tags accumulate hierarchically. The above example would result in: +Tags accumulate hierarchically. The [earlier example](/reference/resource-configs/tags#use-tags-to-run-parts-of-your-project) would result in: | Model | Tags | | -------------------------------- | ------------------------------------- | @@ -218,7 +250,7 @@ Tags accumulate hierarchically. The above example would result in: ### Other resource types -Tags can also be applied to [sources](/docs/build/sources), [exposures](/docs/build/exposures), [exports](/docs/use-dbt-semantic-layer/exports), and even _specific columns_ in a resource. +Tags can also be applied to [sources](/docs/build/sources), [exposures](/docs/build/exposures), and even _specific columns_ in a resource. These resources do not yet support the `config` property, so you'll need to specify the tags as a top-level key instead. @@ -246,15 +278,11 @@ sources: tests: - unique: tags: ['test_level'] - -exports: - - name: export_name - tags: ['export_tag'] - ... ``` + In the example above, the `unique` test would be selected by any of these four tags: ```bash $ dbt test --select tag:top_level From 4cd118c64635203116552b90fe2c7c9f8ea1373b Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Mon, 18 Nov 2024 09:42:56 -0500 Subject: [PATCH 07/13] update rn --- website/docs/docs/dbt-versions/release-notes.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/website/docs/docs/dbt-versions/release-notes.md b/website/docs/docs/dbt-versions/release-notes.md index f7faac30645..d96d5207bd3 100644 --- a/website/docs/docs/dbt-versions/release-notes.md +++ b/website/docs/docs/dbt-versions/release-notes.md @@ -19,12 +19,7 @@ Release notes are grouped by month for both multi-tenant and virtual private clo \* The official release date for this new format of release notes is May 15th, 2024. Historical release notes for prior dates may not reflect all available features released earlier this year or their tenancy availability. ## November 2024 -- **Fix**: This update improves [dbt Semantic Layer Tableau integration](/docs/cloud-integrations/semantic-layer/tableau) making query parsing more reliable. Some key fixes include: - - Error messages for unsupported joins between saved queries and ALL tables. - - Improved handling of queries when multiple tables are selected in a data source. - - Fixed a bug when an IN filter contained a lot of values. - - Better error messaging for queries that can't be parsed correctly. -- **New**: Exports now support [tags](/reference/resource-configs/tags) in dbt Cloud. Tags allow you to categorize your resources and filter them in dbt Cloud. You can add tags to your [exports](/docs/build/saved-queries#configure-exports) in the `semantic_model.yml` file or `dbt_project.yml` file. For example: +- **New**: Exports now support [tags](/reference/resource-configs/tags) in dbt. Tags allow you to categorize your resources and filter them. You can add tags to your [exports](/docs/build/saved-queries#configure-exports) in the `semantic_model.yml` file or `dbt_project.yml` file. For example: ```yml exports: @@ -32,6 +27,12 @@ Release notes are grouped by month for both multi-tenant and virtual private clo tags: ['export_tag'] ... ``` +- **Fix**: This update improves [dbt Semantic Layer Tableau integration](/docs/cloud-integrations/semantic-layer/tableau) making query parsing more reliable. Some key fixes include: + - Error messages for unsupported joins between saved queries and ALL tables. + - Improved handling of queries when multiple tables are selected in a data source. + - Fixed a bug when an IN filter contained a lot of values. + - Better error messaging for queries that can't be parsed correctly. + ## October 2024 From 16faf8cde0cd3659f8d300a81214f7c7be61f7dd Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 10 Dec 2024 10:47:54 +0000 Subject: [PATCH 08/13] Update exports.md --- website/docs/docs/use-dbt-semantic-layer/exports.md | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/website/docs/docs/use-dbt-semantic-layer/exports.md b/website/docs/docs/use-dbt-semantic-layer/exports.md index d5285c8f779..2ecffc508a2 100644 --- a/website/docs/docs/use-dbt-semantic-layer/exports.md +++ b/website/docs/docs/use-dbt-semantic-layer/exports.md @@ -24,10 +24,7 @@ Essentially, exports are like any other table in your data platform — they ## Benefits of exports -The following section explains the main benefits of using exports, including: -- [DRY representation](#dry-representation) -- [Easier changes](#easier-changes) -- [Caching](#caching) +The following section explains the main benefits of using exports: From e2ccfb34d341c3bf3e8262ad02062a2e2164626d Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Wed, 11 Dec 2024 11:27:31 +0000 Subject: [PATCH 09/13] update to saved queries --- website/docs/docs/build/saved-queries.md | 14 ++-- .../docs/docs/dbt-versions/release-notes.md | 12 +-- .../docs/reference/resource-configs/tags.md | 77 ++++++++++--------- 3 files changed, 54 insertions(+), 49 deletions(-) diff --git a/website/docs/docs/build/saved-queries.md b/website/docs/docs/build/saved-queries.md index 68876b65160..840b1ebb95c 100644 --- a/website/docs/docs/build/saved-queries.md +++ b/website/docs/docs/build/saved-queries.md @@ -26,7 +26,7 @@ Note that we use the double colon (::) to indicate whether a parameter is nested | `name` | String | Required | Name of the saved query object. | | `description` | String | Required | A description of the saved query. | | `label` | String | Required | The display name for your saved query. This value will be shown in downstream tools. | -| `config` | String | Optional | Use the [`config`](/reference/resource-properties/config) property to specify configurations for your saved query. Supports `cache`, [`enabled`](/reference/resource-configs/enabled), `export_as`, [`group`](/reference/resource-configs/group), [`meta`](/reference/resource-configs/meta), and [`schema`](/reference/resource-configs/schema) configurations. | +| `config` | String | Optional | Use the [`config`](/reference/resource-properties/config) property to specify configurations for your saved query. Supports `cache`, [`enabled`](/reference/resource-configs/enabled), `export_as`, [`group`](/reference/resource-configs/group), [`meta`](/reference/resource-configs/meta), [`tags`](/reference/resource-configs/tags), and [`schema`](/reference/resource-configs/schema) configurations. | | `config::cache::enabled` | Object | Optional | An object with a sub-key used to specify if a saved query should populate the [cache](/docs/use-dbt-semantic-layer/sl-cache). Accepts sub-key `true` or `false`. Defaults to `false` | | `query_params` | Structure | Required | Contains the query parameters. | | `query_params::metrics` | List or String | Optional | A list of the metrics to be used in the query as specified in the command line interface. | @@ -38,7 +38,6 @@ Note that we use the double colon (::) to indicate whether a parameter is nested | `exports::config::export_as` | String | Required | The type of export to run. Options include table or view currently and cache in the near future. | | `exports::config::schema` | String | Optional | The [schema](/reference/resource-configs/schema) for creating the table or view. This option cannot be used for caching. | | `exports::config::alias` | String | Optional | The table [alias](/reference/resource-configs/alias) used to write to the table or view. This option cannot be used for caching. | -| `exports::config::tags` | String | Optional | Apply a [tag](/reference/resource-configs/tags) (or list of tags) to a resource. Tags help organize and filter resources in dbt. Nested within `exports` and `config`. | @@ -108,7 +107,8 @@ saved_queries: label: Test saved query config: cache: - enabled: true # Or false if you want it disabled by default + [enabled](/reference/resource-configs/enabled): true | false + [tags](/reference/resource-configs/tags): 'my_tag' query_params: metrics: - simple_metric @@ -123,7 +123,6 @@ saved_queries: export_as: table alias: my_export_alias schema: my_export_schema_name - tags: 'my_tag' ``` @@ -243,6 +242,9 @@ The following is an example of a saved query with an export: saved_queries: - name: order_metrics description: Relevant order metrics + config: + tags: + - order_metrics query_params: metrics: - orders @@ -262,9 +264,7 @@ saved_queries: config: export_as: table # Options available: table, view [alias](/reference/resource-configs/alias): my_export_alias # Optional - defaults to Export name - [schema](/reference/resource-configs/schema): my_export_schema_name # Optional - defaults to deployment schema - [tags](/reference/resource-configs/tags): 'my_tag' - + [schema](/reference/resource-configs/schema): my_export_schema_name # Optional - defaults to deployment schema ``` diff --git a/website/docs/docs/dbt-versions/release-notes.md b/website/docs/docs/dbt-versions/release-notes.md index ad8e2d3a9a9..88d3d2b88f2 100644 --- a/website/docs/docs/dbt-versions/release-notes.md +++ b/website/docs/docs/dbt-versions/release-notes.md @@ -20,14 +20,16 @@ Release notes are grouped by month for both multi-tenant and virtual private clo ## December 2024 -- **New**: Exports now support [tags](/reference/resource-configs/tags) in dbt. Tags allow you to categorize your resources and filter them. You can add tags to your [exports](/docs/build/saved-queries#configure-exports) in the `semantic_model.yml` file or `dbt_project.yml` file. For example: +- **New**: Saved queries now support [tags](/reference/resource-configs/tags), which allow you to categorize your resources and filter them. Add tags to your [saved queries](/docs/build/saved-queries) in the `semantic_model.yml` file or `dbt_project.yml` file. For example: + ```yml - exports: - - name: export_name - tags: ['export_tag'] - ... + [saved-queries](/docs/build/saved-queries): + jaffle_shop: + customer_order_metrics: + +tags: order_metrics ``` + - **New**: [Model notifications](/docs/deploy/model-notifications) are now generally available in dbt Cloud. These notifications alert model owners through email about any issues encountered by models and tests as soon as they occur while running a job. - **New**: You can now use your [Azure OpenAI key](/docs/cloud/account-integrations?ai-integration=azure#ai-integrations) (available in beta) to use dbt Cloud features like [dbt Copilot](/docs/cloud/dbt-copilot) and [Ask dbt](/docs/cloud-integrations/snowflake-native-app) . Additionally, you can use your own [OpenAI API key](/docs/cloud/account-integrations?ai-integration=openai#ai-integrations) or use [dbt Labs-managed OpenAI](/docs/cloud/account-integrations?ai-integration=dbtlabs#ai-integrations) key. Refer to [AI integrations](/docs/cloud/account-integrations#ai-integrations) for more information. - **New**: The [`hard_deletes`](/reference/resource-configs/hard-deletes) config gives you more control on how to handle deleted rows from the source. Supported options are `ignore` (default), `invalidate` (replaces the legacy `invalidate_hard_deletes=true`), and `new_record`. Note that `new_record` will create a new metadata column in the snapshot table. diff --git a/website/docs/reference/resource-configs/tags.md b/website/docs/reference/resource-configs/tags.md index e8838a72422..45c4e719059 100644 --- a/website/docs/reference/resource-configs/tags.md +++ b/website/docs/reference/resource-configs/tags.md @@ -20,15 +20,15 @@ datatype: string | [string] ```yml -models: +[models](/reference/model-configs): [](/reference/resource-configs/resource-path): +tags: | [] -snapshots: +[snapshots](/reference/snapshot-configs): [](/reference/resource-configs/resource-path): +tags: | [] -seeds: +[seeds](/reference/seed-configs): [](/reference/resource-configs/resource-path): +tags: | [] @@ -39,19 +39,19 @@ seeds: ```yml -models: +[models](/reference/model-configs): [](/reference/resource-configs/resource-path): +tags: | [] -snapshots: +[snapshots](/reference/snapshot-configs): [](/reference/resource-configs/resource-path): +tags: | [] -seeds: +[seeds](/reference/seed-configs): [](/reference/resource-configs/resource-path): +tags: | [] -exports: +[saved-queries:](/docs/build/saved-queries) [](/reference/resource-configs/resource-path): +tags: | [] @@ -64,38 +64,45 @@ exports: - + -```yml -version: 2 +The following examples show how to add tags to dbt resources in YAML files. Replace `resource_type` with `models`, `snapshots`, `seeds`, or `saved_queries` as appropriate. + -models: - - name: model_name - config: - tags: | [] + + +The following examples show how to add tags to dbt resources in YAML files. Replace `resource_type` with `models`, `snapshots`, or `seeds` as appropriate. + + + +```yaml +resource_type: + - name: resource_name + config: + tags: string + # Optional: Add specific properties for models columns: - name: column_name - tags: [] + tags: string tests: - : + test-name: config: - tags: | [] + tags: string ``` - -```jinja - + +```sql {{ config( tags="" | [""] ) }} - ``` + @@ -191,22 +198,22 @@ seeds: -### Apply tags to exports +### Apply tags to saved queries -Applying tags to exports is only available in dbt Core versions 1.9 and later. +Applying tags to saved queries is only available in dbt Core versions 1.9 and later. -This following example shows how to apply tags to an export in the `dbt_project.yml` file. The export is then tagged with `order_metrics`. +This following example shows how to apply a tag to a saved query in the `dbt_project.yml` file. The saved query is then tagged with `order_metrics`. ```yml -[exports](/docs/build/saved-queries#configure-exports): +[saved-queries](/docs/build/saved-queries): jaffle_shop: customer_order_metrics: +tags: order_metrics @@ -214,23 +221,19 @@ This following example shows how to apply tags to an export in the `dbt_project. -The second example shows how to apply tags to an export in the `semantic_model.yml` file. The export is then tagged with `order_metrics` and `hourly`. +The second example shows how to apply multiple tags to a saved query in the `semantic_model.yml` file. The saved query is then tagged with `order_metrics` and `hourly`. ```yaml saved_queries: - - name: order_metrics - ... - exports: - - name: hourly_order_metrics - config: - alias: my_export_alias - export_as: table - schema: my_export_schema_name - tags: - - order_metrics - - hourly + - name: test_saved_query + description: "{{ doc('saved_query_description') }}" + label: Test saved query + config: + tags: + - order_metrics + - hourly ``` From 952374035b02b072216dfefcc7bb499b5edbb74e Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Mon, 23 Dec 2024 12:07:35 +0000 Subject: [PATCH 10/13] feedback --- website/docs/reference/resource-configs/tags.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/website/docs/reference/resource-configs/tags.md b/website/docs/reference/resource-configs/tags.md index 45c4e719059..7e0016f0d35 100644 --- a/website/docs/reference/resource-configs/tags.md +++ b/website/docs/reference/resource-configs/tags.md @@ -22,7 +22,7 @@ datatype: string | [string] [models](/reference/model-configs): [](/reference/resource-configs/resource-path): - +tags: | [] + +tags: | [] # Supports single strings or list of strings [snapshots](/reference/snapshot-configs): [](/reference/resource-configs/resource-path): @@ -41,7 +41,7 @@ datatype: string | [string] [models](/reference/model-configs): [](/reference/resource-configs/resource-path): - +tags: | [] + +tags: | [] # Supports single strings or list of strings [snapshots](/reference/snapshot-configs): [](/reference/resource-configs/resource-path): @@ -80,15 +80,16 @@ The following examples show how to add tags to dbt resources in YAML files. Repl resource_type: - name: resource_name config: - tags: string - # Optional: Add specific properties for models + tags: | [] # Supports single strings or list of strings + # Optional: Add the following specific properties for models columns: - name: column_name - tags: string + tags: | [] tests: test-name: config: - tags: string + tags: "single-string" # Supports single string + tags: ["string-1", "string-2"] # Supports list of strings ``` From 3e561f12b0b1e35a531e0c17363218d3beadf64b Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 24 Dec 2024 10:53:36 +0000 Subject: [PATCH 11/13] Update tags.md add command examples --- website/docs/reference/resource-configs/tags.md | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/website/docs/reference/resource-configs/tags.md b/website/docs/reference/resource-configs/tags.md index 7e0016f0d35..27be1f71766 100644 --- a/website/docs/reference/resource-configs/tags.md +++ b/website/docs/reference/resource-configs/tags.md @@ -114,6 +114,7 @@ Apply a tag (or list of tags) to a resource. These tags can be used as part of the [resource selection syntax](/reference/node-selection/syntax), when running the following commands: - `dbt run --select tag:my_tag` +- `dbt build --select tag:my_tag` - `dbt seed --select tag:my_tag` - `dbt snapshot --select tag:my_tag` - `dbt test --select tag:my_tag` (indirectly runs all tests associated with the models that are tagged) @@ -222,6 +223,13 @@ This following example shows how to apply a tag to a saved query in the `dbt_pro +Then, run part of your project like so: + +``` +# Run all resources tagged "order_metrics" +dbt run --select tag:order_metrics +``` + The second example shows how to apply multiple tags to a saved query in the `semantic_model.yml` file. The saved query is then tagged with `order_metrics` and `hourly`. @@ -236,8 +244,15 @@ saved_queries: - order_metrics - hourly ``` - + + +Then, run part of your project like so: + +``` +# Run all resources tagged "order_metrics" and "hourly" +dbt build --select tag:order_metrics tag:hourly +``` ## Usage notes From f98a1a70fb5f4947f5cdd177572381c459155457 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Tue, 24 Dec 2024 12:21:45 +0000 Subject: [PATCH 12/13] natalies feedback --- .../docs/reference/resource-configs/tags.md | 31 ++++++++++--------- 1 file changed, 17 insertions(+), 14 deletions(-) diff --git a/website/docs/reference/resource-configs/tags.md b/website/docs/reference/resource-configs/tags.md index 27be1f71766..b77c1472032 100644 --- a/website/docs/reference/resource-configs/tags.md +++ b/website/docs/reference/resource-configs/tags.md @@ -164,14 +164,14 @@ select ... -Then, run part of your project like so: +Run resources with specific tags (or exclude resources with specific tags) with the following commands: -``` +```shell # Run all models tagged "daily" -$ dbt run --select tag:daily + dbt run --select tag:daily # Run all models tagged "daily", except those that are tagged hourly -$ dbt run --select tag:daily --exclude tag:hourly + dbt run --select tag:daily --exclude tag:hourly ``` ### Apply tags to seeds @@ -204,7 +204,10 @@ seeds: +:::tip Upgrade to dbt Core 1.9 + Applying tags to saved queries is only available in dbt Core versions 1.9 and later. +::: @@ -223,11 +226,11 @@ This following example shows how to apply a tag to a saved query in the `dbt_pro -Then, run part of your project like so: +Then run resources with a specific tag with the following commands: -``` +```shell # Run all resources tagged "order_metrics" -dbt run --select tag:order_metrics + dbt run --select tag:order_metrics ``` The second example shows how to apply multiple tags to a saved query in the `semantic_model.yml` file. The saved query is then tagged with `order_metrics` and `hourly`. @@ -247,11 +250,11 @@ saved_queries: -Then, run part of your project like so: +Run resources with multiple tags with the following commands: -``` +```shell # Run all resources tagged "order_metrics" and "hourly" -dbt build --select tag:order_metrics tag:hourly + dbt build --select tag:order_metrics tag:hourly ``` @@ -304,8 +307,8 @@ sources: In the example above, the `unique` test would be selected by any of these four tags: ```bash -$ dbt test --select tag:top_level -$ dbt test --select tag:table_level -$ dbt test --select tag:column_level -$ dbt test --select tag:test_level +dbt test --select tag:top_level +dbt test --select tag:table_level +dbt test --select tag:column_level +dbt test --select tag:test_level ``` From 642912c84934c807d3cbaf19f077094b48fcecaf Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Tue, 24 Dec 2024 12:22:57 +0000 Subject: [PATCH 13/13] grammar fix --- website/docs/reference/resource-configs/tags.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/website/docs/reference/resource-configs/tags.md b/website/docs/reference/resource-configs/tags.md index b77c1472032..c222df8c1ae 100644 --- a/website/docs/reference/resource-configs/tags.md +++ b/website/docs/reference/resource-configs/tags.md @@ -164,7 +164,7 @@ select ... -Run resources with specific tags (or exclude resources with specific tags) with the following commands: +Run resources with specific tags (or exclude resources with specific tags) using the following commands: ```shell # Run all models tagged "daily" @@ -226,7 +226,7 @@ This following example shows how to apply a tag to a saved query in the `dbt_pro -Then run resources with a specific tag with the following commands: +Then run resources with a specific tag using the following commands: ```shell # Run all resources tagged "order_metrics" @@ -250,7 +250,7 @@ saved_queries: -Run resources with multiple tags with the following commands: +Run resources with multiple tags using the following commands: ```shell # Run all resources tagged "order_metrics" and "hourly"