Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

saved queries support tags #6632

Merged
merged 27 commits into from
Dec 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
eb93839
add tags to exports
mirnawong1 Nov 6, 2024
7443521
fix link
mirnawong1 Nov 6, 2024
44b9248
fix link
mirnawong1 Nov 6, 2024
53c0351
update example
mirnawong1 Nov 6, 2024
c9a0820
fix link
mirnawong1 Nov 6, 2024
8162530
update tags
mirnawong1 Nov 18, 2024
4cd118c
update rn
mirnawong1 Nov 18, 2024
c891534
Merge branch 'current' into add-tags
mirnawong1 Dec 10, 2024
16faf8c
Update exports.md
mirnawong1 Dec 10, 2024
a5ffb73
Merge branch 'current' into add-tags
mirnawong1 Dec 11, 2024
85afe10
Merge branch 'current' into add-tags
mirnawong1 Dec 11, 2024
359ed5e
Merge branch 'current' into add-tags
mirnawong1 Dec 11, 2024
e2ccfb3
update to saved queries
mirnawong1 Dec 11, 2024
f1c4a10
resovle
mirnawong1 Dec 11, 2024
66d4eb3
Merge branch 'current' into add-tags
mirnawong1 Dec 11, 2024
33e1598
Merge branch 'current' into add-tags
mirnawong1 Dec 20, 2024
08a5f20
Merge branch 'current' into add-tags
mirnawong1 Dec 20, 2024
b9d57c8
Merge branch 'current' into add-tags
mirnawong1 Dec 23, 2024
9523740
feedback
mirnawong1 Dec 23, 2024
4557104
Merge branch 'current' into add-tags
mirnawong1 Dec 24, 2024
7b73912
Merge branch 'current' into add-tags
mirnawong1 Dec 24, 2024
c38c6e1
Merge branch 'current' into add-tags
mirnawong1 Dec 24, 2024
3e561f1
Update tags.md
mirnawong1 Dec 24, 2024
a90c932
Merge branch 'current' into add-tags
mirnawong1 Dec 24, 2024
79a3553
Merge branch 'current' into add-tags
mirnawong1 Dec 24, 2024
f98a1a7
natalies feedback
mirnawong1 Dec 24, 2024
642912c
grammar fix
mirnawong1 Dec 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
116 changes: 103 additions & 13 deletions website/docs/docs/build/saved-queries.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,32 @@ To create a saved query, refer to the following table parameters.
:::tip
Note that we use the double colon (::) to indicate whether a parameter is nested within another parameter. So for example, `query_params::metrics` means the `metrics` parameter is nested under `query_params`.
:::

<!-- For versions 1.9 and higher -->
<VersionBlock firstVersion="1.9">

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't review this as I'm not sure what has changed and it doesn't seem related to tags

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good shout! just added that tags are supported in 1.9+ under the config value:
Screenshot 2024-12-23 at 11 43 39


| Parameter | Type | Required | Description |
|-------|---------|----------|----------------|
| `name` | String | Required | Name of the saved query object. |
| `description` | String | Required | A description of the saved query. |
| `label` | String | Required | The display name for your saved query. This value will be shown in downstream tools. |
| `config` | String | Optional | Use the [`config`](/reference/resource-properties/config) property to specify configurations for your saved query. Supports `cache`, [`enabled`](/reference/resource-configs/enabled), `export_as`, [`group`](/reference/resource-configs/group), [`meta`](/reference/resource-configs/meta), [`tags`](/reference/resource-configs/tags), and [`schema`](/reference/resource-configs/schema) configurations. |
| `config::cache::enabled` | Object | Optional | An object with a sub-key used to specify if a saved query should populate the [cache](/docs/use-dbt-semantic-layer/sl-cache). Accepts sub-key `true` or `false`. Defaults to `false` |
| `query_params` | Structure | Required | Contains the query parameters. |
| `query_params::metrics` | List or String | Optional | A list of the metrics to be used in the query as specified in the command line interface. |
| `query_params::group_by` | List or String | Optional | A list of the Entities and Dimensions to be used in the query, which include the `Dimension` or `TimeDimension`. |
| `query_params::where` | List or String | Optional | A list of strings that may include the `Dimension` or `TimeDimension` objects. |
| `exports` | List or Structure | Optional | A list of exports to be specified within the exports structure. |
| `exports::name` | String | Required | Name of the export object. |
| `exports::config` | List or Structure | Required | A [`config`](/reference/resource-properties/config) property for any parameters specifying the export. |
| `exports::config::export_as` | String | Required | The type of export to run. Options include table or view currently and cache in the near future. |
| `exports::config::schema` | String | Optional | The [schema](/reference/resource-configs/schema) for creating the table or view. This option cannot be used for caching. |
| `exports::config::alias` | String | Optional | The table [alias](/reference/resource-configs/alias) used to write to the table or view. This option cannot be used for caching. |

</VersionBlock>

<!-- For versions 1.8 and higher -->
<VersionBlock firstVersion="1.8">
<VersionBlock firstVersion="1.8" lastVersion="1.8">

| Parameter | Type | Required | Description |
|-------|---------|----------|----------------|
Expand All @@ -33,15 +57,15 @@ Note that we use the double colon (::) to indicate whether a parameter is nested
| `query_params::where` | List or String | Optional | A list of strings that may include the `Dimension` or `TimeDimension` objects. |
| `exports` | List or Structure | Optional | A list of exports to be specified within the exports structure. |
| `exports::name` | String | Required | Name of the export object. |
| `exports::config` | List or Structure | Required | A config section for any parameters specifying the export. |
| `exports::config` | List or Structure | Required | A [`config`](/reference/resource-properties/config) property for any parameters specifying the export. |
| `exports::config::export_as` | String | Required | The type of export to run. Options include table or view currently and cache in the near future. |
| `exports::config::schema` | String | Optional | The schema for creating the table or view. This option cannot be used for caching. |
| `exports::config::alias` | String | Optional | The table alias used to write to the table or view. This option cannot be used for caching. |
| `exports::config::alias` | String | Optional | The table alias used to write to the table or view. This option cannot be used for caching. |

</VersionBlock>

<!-- For versions 1.7 and lower-->
<VersionBlock lastVersion="1.7">
<VersionBlock firstVersion="1.7" lastVersion="1.7">

| Parameter | Type | Required | Description |
|-------|---------|----------|----------------|
Expand All @@ -54,7 +78,7 @@ Note that we use the double colon (::) to indicate whether a parameter is nested
| `query_params::where` | List or String | Optional | Conditions nested with the `query_params`: a list of strings that may include the `Dimension` or `TimeDimension` objects. |
| `exports` | List or Structure | Optional | A list of exports to be specified within the exports structure. |
| `exports::name` | String | Required | Name of export object, nested within `exports`. |
| `exports::config` | List or Structure | Required | A config section for any parameters specifying the export, nested within `exports`. |
| `exports::config` | List or Structure | Required | A [`config`](/reference/resource-properties/config) property for any parameters specifying the export, nested within `exports`. |
| `exports::config::export_as` | String | Required | Specifies the type of export: table, view, or upcoming cache options. Nested within `exports` and `config`. |
| `exports::config::schema` | String | Optional | Schema for creating the table or view, not applicable for caching. Nested within `exports` and `config`. |
| `exports::config::alias` | String | Optional | Table alias used to write to the table or view. This option can't be used for caching. Nested within `exports` and `config`. |
Expand All @@ -69,13 +93,12 @@ Use saved queries to define and manage common Semantic Layer queries in YAML, in

In your saved query config, you can also leverage [caching](/docs/use-dbt-semantic-layer/sl-cache) with the dbt Cloud job scheduler to cache common queries, speed up performance, and reduce compute costs.

<!-- For versions 1.8 and higher -->

<VersionBlock firstVersion="1.8">
<!-- For versions 1.9 and higher -->

In the following example, you can set the saved query in the `semantic_model.yml` file:

<File name='semantic_model.yml'>
<VersionBlock firstVersion="1.9">

```yaml
saved_queries:
Expand All @@ -84,7 +107,8 @@ saved_queries:
label: Test saved query
config:
cache:
enabled: true # Or false if you want it disabled by default
[enabled](/reference/resource-configs/enabled): true | false
[tags](/reference/resource-configs/tags): 'my_tag'
query_params:
metrics:
- simple_metric
Expand All @@ -96,12 +120,44 @@ saved_queries:
exports:
- name: my_export
config:
export_as: table
alias: my_export_alias
schema: my_export_schema_name
```
</VersionBlock>

<!-- For versions 1.8 and higher -->
<VersionBlock firstVersion="1.8" lastVersion="1.8">

```yaml
saved_queries:
- name: test_saved_query
description: "{{ doc('saved_query_description') }}"
label: Test saved query
config:
cache:
enabled: true # Or false if you want it disabled by default
query_params:
metrics:
- simple_metric
group_by:
- "Dimension('user__ds')"
where:
- "{{ Dimension('user__ds', 'DAY') }} <= now()"
- "{{ Dimension('user__ds', 'DAY') }} >= '2023-01-01'"
exports:
- name: my_export
config:
export_as: table
alias: my_export_alias
schema: my_export_schema_name
```

</VersionBlock>
</File>

<VersionBlock firstVersion="1.8">

Note, that you can set `export_as` to both the saved query and the exports [config](/reference/resource-properties/config), with the exports config value taking precedence. If a key isn't set in the exports config, it will inherit the saved query config value.

#### Where clause
Expand All @@ -121,7 +177,6 @@ filter: |
filter: |
{{ Metric('metric_name', group_by=['entity_name']) }}
```

</VersionBlock>

<!-- For versions 1.7 and lower-->
Expand All @@ -147,8 +202,8 @@ saved_queries:
exports:
- name: my_export
config:
alias: my_export_alias
export_as: table
alias: my_export_alias
schema: my_export_schema_name
```
</File>
Expand Down Expand Up @@ -181,11 +236,15 @@ Once you've configured your saved query and set the foundation block, you can no
The following is an example of a saved query with an export:

<File name='semantic_model.yml'>
<VersionBlock firstVersion="1.9">

```yaml
saved_queries:
- name: order_metrics
description: Relevant order metrics
config:
tags:
- order_metrics
query_params:
metrics:
- orders
Expand All @@ -204,9 +263,40 @@ saved_queries:
- name: order_metrics
config:
export_as: table # Options available: table, view
schema: YOUR_SCHEMA # Optional - defaults to deployment schema
alias: SOME_TABLE_NAME # Optional - defaults to Export name
[alias](/reference/resource-configs/alias): my_export_alias # Optional - defaults to Export name
[schema](/reference/resource-configs/schema): my_export_schema_name # Optional - defaults to deployment schema
```
</VersionBlock>

<VersionBlock lastVersion="1.8">

```yaml
saved_queries:
- name: order_metrics
description: Relevant order metrics
query_params:
metrics:
- orders
- large_order
- food_orders
- order_total
group_by:
- Entity('order_id')
- TimeDimension('metric_time', 'day')
- Dimension('customer__customer_name')
- ... # Additional group_by
where:
- "{{TimeDimension('metric_time')}} > current_timestamp - interval '1 week'"
- ... # Additional where clauses
exports:
- name: order_metrics
config:
export_as: table # Options available: table, view
schema: my_export_schema_name # Optional - defaults to deployment schema
alias: my_export_alias # Optional - defaults to Export name
```
</VersionBlock>

</File>

## Run exports
Expand Down
12 changes: 12 additions & 0 deletions website/docs/docs/dbt-versions/release-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,16 @@ Release notes are grouped by month for both multi-tenant and virtual private clo

## December 2024

- **New**: Saved queries now support [tags](/reference/resource-configs/tags), which allow you to categorize your resources and filter them. Add tags to your [saved queries](/docs/build/saved-queries) in the `semantic_model.yml` file or `dbt_project.yml` file. For example:
<File name='dbt_project.yml'>

```yml
[saved-queries](/docs/build/saved-queries):
jaffle_shop:
customer_order_metrics:
+tags: order_metrics
```
</File>
- **New**: [Dimensions](/reference/resource-configs/meta) now support the `meta` config property in [dbt Cloud "Latest" release track](/docs/dbt-versions/cloud-release-tracks) and from dbt Core 1.9. You can add metadata to your dimensions to provide additional context and information about the dimension. Refer to [meta](/reference/resource-configs/meta) for more information.
- **New**: [Auto exposures](/docs/collaborate/auto-exposures) are now generally available to dbt Cloud Enterprise plans. Auto-exposures integrate natively with Tableau (Power BI coming soon) and auto-generate downstream lineage in dbt Explorer for a richer experience.
- **New**: The dbt Semantic Layer supports Sigma as a [partner integration](/docs/cloud-integrations/avail-sl-integrations), available in Preview. Refer to [Sigma](https://help.sigmacomputing.com/docs/configure-a-dbt-semantic-layer-integration) for more information.
Expand All @@ -31,6 +41,7 @@ Release notes are grouped by month for both multi-tenant and virtual private clo
- **New**: The [`hard_deletes`](/reference/resource-configs/hard-deletes) config gives you more control on how to handle deleted rows from the source. Supported options are `ignore` (default), `invalidate` (replaces the legacy `invalidate_hard_deletes=true`), and `new_record`. Note that `new_record` will create a new metadata column in the snapshot table.

## November 2024

- **Enhancement**: Data health signals in dbt Explorer are now available for Exposures, providing a quick view of data health while browsing resources. To view trust signal icons, go to dbt Explorer and click **Exposures** under the **Resource** tab. Refer to [Data health signals for resources](/docs/collaborate/data-health-signals) for more info.
- **Bug**: Identified and fixed an error with Semantic Layer queries that take longer than 10 minutes to complete.
- **Fix**: Job environment variable overrides in credentials are now respected for Exports. Previously, they were ignored.
Expand All @@ -46,6 +57,7 @@ Release notes are grouped by month for both multi-tenant and virtual private clo
- Better error messaging for queries that can't be parsed correctly.
- **Enhancement**: The dbt Semantic Layer supports creating new credentials for users who don't have permissions to create service tokens. In the **Credentials & service tokens** side panel, the **+Add Service Token** option is unavailable for those users who don't have permission. Instead, the side panel displays a message indicating that the user doesn't have permission to create a service token and should contact their administration. Refer to [Set up dbt Semantic Layer](/docs/use-dbt-semantic-layer/setup-sl) for more details.


## October 2024

<Expandable alt_header="Coalesce 2024 announcements">
Expand Down
15 changes: 8 additions & 7 deletions website/docs/docs/use-dbt-semantic-layer/exports.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,21 +24,22 @@ Essentially, exports are like any other table in your data platform &mdash; they

## Benefits of exports

The following section explains the main benefits of using exports, including:
- [DRY representation](#dry-representation)
- [Easier changes](#easier-changes)
- [Caching](#caching)
The following section explains the main benefits of using exports:

#### DRY representation
<Expandable alt_header="DRY representation">

Currently, creating tables often involves generating tens, hundreds, or even thousands of tables that denormalize data into summary or metric mart tables. The main benefit of exports is creating a "Don't Repeat Yourself (DRY)" representation of the logic to construct each metric, dimension, join, filter, and so on. This allows you to reuse those components for long-term scalability, even if you're replacing manually written SQL models with references to the metrics or dimensions in saved queries.
</Expandable>

#### Easier changes
<Expandable alt_header="Easier changes">

Exports ensure that changes to metrics and dimensions are made in one place and then cascade to those various destinations seamlessly. This prevents the problem of needing to update a metric across every model that references that same concept.
</Expandable>

<Expandable alt_header="Caching">

#### Caching
Use exports to pre-populate the cache, so that you're pre-computing what you need to serve users through the dynamic Semantic Layer APIs.
</Expandable>

#### Considerations

Expand Down
Loading
Loading