From 132668e3e7fa79338196f4ebddd08f25c453aaa6 Mon Sep 17 00:00:00 2001 From: rpourzand Date: Wed, 20 Sep 2023 15:50:17 -0700 Subject: [PATCH 1/9] Update sl-jdbc.md Draft PR --- website/docs/docs/dbt-cloud-apis/sl-jdbc.md | 29 ++++++++++++++++----- 1 file changed, 22 insertions(+), 7 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md index b50dd99ec75..0106b2054c0 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md +++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md @@ -165,9 +165,9 @@ To query metric values, here are the following parameters that are available: | `metrics` | The metric name as defined in your dbt metric configuration | `metrics=['revenue']` | Required | | `group_by` | Dimension names or entities to group by. We require a reference to the entity of the dimension (other than for the primary time dimension), which is pre-appended to the front of the dimension name with a double underscore. | `group_by=['user__country', 'metric_time']` | Optional | | `grain` | A parameter specific to any time dimension and changes the grain of the data from the default for the metric. | `group_by=[Dimension('metric_time')`
`grain('week\|day\|month\|quarter\|year')]` | Optional | -| `where` | A where clause that allows you to filter on dimensions and entities using parameters - comes with `TimeDimension`, `Dimension`, and `Entity` objects. Granularity is required with `TimeDimension` | `"{{ where=Dimension('customer__country') }} = 'US')"` | Optional | +| `where` | A where clause that allows you to filter on dimensions and entities using parameters. This takes a filter list OR string. Inputs come with `Dimension`, and `Entity` objects. Granularity is required if the `Dimension` is a time dimension | `"{{ where=Dimension('customer__country') }} = 'US')"` | Optional | | `limit` | Limit the data returned | `limit=10` | Optional | -|`order` | Order the data returned | `order_by=['-order_gross_profit']` (remove `-` for ascending order) | Optional | +|`order` | Order the data returned | `order_by=['-order_gross_profit']` (remove `-` for ascending order). | Optional | | `explain` | If true, returns generated SQL for the data platform but does not execute | `explain=True` | Optional | @@ -244,13 +244,13 @@ select * from {{ Where filters in API allow for a filter list or string. We recommend using the filter list for production applications as this format will realize all benefits from the where possible. -Where filters have the following components that you can use: +Where Filters have a few objects that you can use: - `Dimension()` - This is used for any categorical or time dimensions. If used for a time dimension, granularity is required - `Dimension('metric_time').grain('week')` or `Dimension('customer__country')` -- `TimeDimension()` - This is used for all time dimensions and requires a granularity argument - `TimeDimension('metric_time', 'MONTH)` +- `Entity()` - used for entities like primary and foreign keys - `Entity('order_id')` -- `Entity()` - This is used for entities like primary and foreign keys - `Entity('order_id')` +Note: If you prefer a more strongly typed `where` clause, you can optionally use the `TimeDimension` feature. This helps separate out categorical dimensions from time-related ones. The `TimeDimesion` input takes the time dimension name and also requires granularity, like this: `TimeDimension('metric_time', 'MONTH')`. Use the following example to query using a `where` filter with the string format: @@ -259,7 +259,7 @@ Use the following example to query using a `where` filter with the string format select * from {{ semantic_layer.query(metrics=['food_order_amount', 'order_gross_profit'], group_by=[Dimension('metric_time').grain('month'),'customer__customer_type'], -where="{{ TimeDimension('metric_time', 'MONTH') }} >= '2017-03-09' AND {{ Dimension('customer__customer_type' }} in ('new') AND {{ Entity('order_id') }} = 10") +where="{{ Dimension('metric_time').grain('month') }} >= '2017-03-09' AND {{ Dimension('customer__customer_type' }} in ('new') AND {{ Entity('order_id') }} = 10") }} ``` @@ -269,7 +269,7 @@ Use the following example to query using a `where` filter with a filter list for select * from {{ semantic_layer.query(metrics=['food_order_amount', 'order_gross_profit'], group_by=[Dimension('metric_time').grain('month'),'customer__customer_type'], -where=[{{ TimeDimension('metric_time', 'MONTH')}} >= '2017-03-09', {{ Dimension('customer__customer_type' }} in ('new'), {{ Entity('order_id') }} = 10]) +where=[{{ Dimension('metric_time').grain('month') }} >= '2017-03-09', {{ Dimension('customer__customer_type' }} in ('new'), {{ Entity('order_id') }} = 10]) }} ``` @@ -285,6 +285,21 @@ semantic_layer.query(metrics=['food_order_amount', 'order_gross_profit'], order_by=['order_gross_profit']) }} ``` + +Note that for `order_by`, if you are operating on the object (e.g. changing granularity), you should use the following syntax: + +```bash +... +order_by=[Dimension('metric_time').grain('month')] +``` + +Descending: +... +``` +order_by=[Dimension('metric_time').grain('month').descending(true)] +``` + + ### Query with explain keyword Use the following example to query using a `explain` keyword: From 1cc4c030a32b080e385227aee057703aa7ba7c5d Mon Sep 17 00:00:00 2001 From: rpourzand Date: Wed, 20 Sep 2023 15:51:51 -0700 Subject: [PATCH 2/9] Update sl-jdbc.md fixing typo --- website/docs/docs/dbt-cloud-apis/sl-jdbc.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md index 0106b2054c0..cb705296a62 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md +++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md @@ -291,11 +291,10 @@ Note that for `order_by`, if you are operating on the object (e.g. changing gran ```bash ... order_by=[Dimension('metric_time').grain('month')] -``` +``` -Descending: -... ``` +... order_by=[Dimension('metric_time').grain('month').descending(true)] ``` From 438568b5881f19755a707befeb95360fb063997e Mon Sep 17 00:00:00 2001 From: rpourzand Date: Fri, 22 Sep 2023 10:41:45 -0700 Subject: [PATCH 3/9] Update website/docs/docs/dbt-cloud-apis/sl-jdbc.md Co-authored-by: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> --- website/docs/docs/dbt-cloud-apis/sl-jdbc.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md index cb705296a62..d96f03912a1 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md +++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md @@ -286,7 +286,7 @@ semantic_layer.query(metrics=['food_order_amount', 'order_gross_profit'], }} ``` -Note that for `order_by`, if you are operating on the object (e.g. changing granularity), you should use the following syntax: +When using `order_by` and making changes to the object (like changing granularity), use this syntax: ```bash ... From 85f17721f14f0b07477b305a94d221d2206cf98c Mon Sep 17 00:00:00 2001 From: rpourzand Date: Fri, 22 Sep 2023 10:43:08 -0700 Subject: [PATCH 4/9] Update sl-jdbc.md Changing wording per Devon's recommendation --- website/docs/docs/dbt-cloud-apis/sl-jdbc.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md index d96f03912a1..dedd69648ae 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md +++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md @@ -250,7 +250,7 @@ Where Filters have a few objects that you can use: - `Entity()` - used for entities like primary and foreign keys - `Entity('order_id')` -Note: If you prefer a more strongly typed `where` clause, you can optionally use the `TimeDimension` feature. This helps separate out categorical dimensions from time-related ones. The `TimeDimesion` input takes the time dimension name and also requires granularity, like this: `TimeDimension('metric_time', 'MONTH')`. +Note: If you prefer a more explicit path to create the `where` clause, you can optionally use the `TimeDimension` feature. This helps separate out categorical dimensions from time-related ones. The `TimeDimesion` input takes the time dimension name and also requires granularity, like this: `TimeDimension('metric_time', 'MONTH')`. Use the following example to query using a `where` filter with the string format: From 690e5bccbdd3a238207743c7ee69852404b8341a Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Mon, 25 Sep 2023 15:33:51 +0100 Subject: [PATCH 5/9] Update website/docs/docs/dbt-cloud-apis/sl-jdbc.md --- website/docs/docs/dbt-cloud-apis/sl-jdbc.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md index dedd69648ae..5c31f37f219 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md +++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md @@ -248,7 +248,7 @@ Where Filters have a few objects that you can use: - `Dimension()` - This is used for any categorical or time dimensions. If used for a time dimension, granularity is required - `Dimension('metric_time').grain('week')` or `Dimension('customer__country')` -- `Entity()` - used for entities like primary and foreign keys - `Entity('order_id')` +- `Entity()` - Used for entities like primary and foreign keys - `Entity('order_id')` Note: If you prefer a more explicit path to create the `where` clause, you can optionally use the `TimeDimension` feature. This helps separate out categorical dimensions from time-related ones. The `TimeDimesion` input takes the time dimension name and also requires granularity, like this: `TimeDimension('metric_time', 'MONTH')`. From 904d209338411413b74ff47571458b531c6ffc06 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Mon, 2 Oct 2023 10:38:48 +0100 Subject: [PATCH 6/9] update tabs --- website/docs/docs/dbt-cloud-apis/sl-jdbc.md | 19 ++++++++++++------- website/docs/guides/migration/sl-migration.md | 4 ++-- 2 files changed, 14 insertions(+), 9 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md index 5c31f37f219..47ff21f316e 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md +++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md @@ -5,7 +5,6 @@ description: "Integrate and use the JDBC API to query your metrics." tags: [Semantic Layer, API] --- - import LegacyInfo from '/snippets/_legacy-sl-callout.md'; @@ -57,11 +56,13 @@ jdbc:arrow-flight-sql://semantic-layer.cloud.getdbt.com:443?&environmentId=20233 ## Querying the API for metric metadata -The Semantic Layer JDBC API has built-in metadata calls which can provide a user with information about their metrics and dimensions. Here are some metadata commands and examples: +The Semantic Layer JDBC API has built-in metadata calls which can provide a user with information about their metrics and dimensions. + +Refer to the following tabs for metadata commands and examples: - + Use this query to fetch all defined metrics in your dbt project: @@ -72,7 +73,7 @@ select * from {{ ``` - + Use this query to fetch all dimensions for a metric. @@ -85,7 +86,7 @@ select * from {{ - + Use this query to fetch dimension values for one or multiple metrics and single dimension. @@ -98,7 +99,7 @@ semantic_layer.dimension_values(metrics=['food_order_amount'], group_by=['custom - + Use this query to fetch queryable granularities for a list of metrics. This API request allows you to only show the time granularities that make sense for the primary time dimension of the metrics (such as `metric_time`), but if you want queryable granularities for other time dimensions, you can use the `dimensions()` call, and find the column queryable_granularities. @@ -111,6 +112,9 @@ select * from {{ + + + @@ -142,9 +146,10 @@ select NAME, QUERYABLE_GRANULARITIES from {{ - + It may be useful in your application to expose the names of the time dimensions that represent `metric_time` or the common thread across all metrics. + You can first query the `metrics()` argument to fetch a list of measures, then use the `measures()` call which will return the name(s) of the time dimensions that make up metric time. ```bash diff --git a/website/docs/guides/migration/sl-migration.md b/website/docs/guides/migration/sl-migration.md index baa7ae4a567..feb10154bee 100644 --- a/website/docs/guides/migration/sl-migration.md +++ b/website/docs/guides/migration/sl-migration.md @@ -65,8 +65,8 @@ This step is only relevant to users who want the legacy and new semantic layer t 1. Create a new deployment environment in dbt Cloud and set the dbt version to 1.6 or higher. 2. Choose `Only run on a custom branch` and point to the branch that has the updated metric definition 3. Set the deployment schema to a temporary migration schema, such as `tmp_sl_migration`. Optional, you can create a new database for the migration. -4. Create a job to parse your project, such as `dbt parse`, and run it. Make sure this job succeeds, There needs to be a successful job in your environment in order to set up the semantic layer -5. In Account Settings > Projects > Project details click `Configure the Semantic Layer`. Under **Environment**select the deployment environment you created in the previous step. Save your configuration. +4. Create a job to parse your project, such as `dbt parse`, and run it. Make sure this job succeeds, there needs to be a successful job in your environment in order to set up the semantic layer +5. In Account Settings > Projects > Project details click `Configure the Semantic Layer`. Under **Environment**, select the deployment environment you created in the previous step. Save your configuration. 6. In the Project details page, click `Generate service token` and grant it `Semantic Layer Only` and `Metadata Only` permissions. Save this token securely - you will need it to connect to the semantic layer. At this point, both the new semantic layer and the old semantic layer will be running. The new semantic layer will be pointing at your migration branch with the updated metrics definitions. From 0ba851022cce6955ba6e431ca804de6ea118b2f0 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Mon, 2 Oct 2023 15:16:25 +0100 Subject: [PATCH 7/9] clarify integrations --- website/snippets/_sl-partner-links.md | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/website/snippets/_sl-partner-links.md b/website/snippets/_sl-partner-links.md index e9cc6af3564..2f62974d810 100644 --- a/website/snippets/_sl-partner-links.md +++ b/website/snippets/_sl-partner-links.md @@ -1,11 +1,23 @@ -The dbt Semantic Layer integrations are capable of querying dbt metrics, importing definitions, surfacing the underlying data in partner tools, and more. These are the following tools that integrate with the dbt Semantic Layer: +## Integrations -1. **Mode** — To learn more about integrating with Mode, check out their [documentation](https://mode.com/help/articles/supported-databases/#dbt-semantic-layer) for more info. -2. **Hex** — To learn more about integrating with Hex, check out their [documentation](https://learn.hex.tech/docs/connect-to-data/data-connections/dbt-integration#dbt-semantic-layer-integration) for more info. Additionally, refer to [dbt Semantic Layer cells](https://learn.hex.tech/docs/logic-cell-types/transform-cells/dbt-metrics-cells) to set up SQL cells in Hex. -3. **Google Sheets** — Google Sheets integration coming soon. -4. **Tools that allows you to write SQL** — They must meet one of the two criteria: - * Supports a generic JDBC driver option (such as DataGrip) or - * Supports Dremio and uses ArrowFlightSQL driver version 12.0.0 or higher. +The dbt Semantic Layer integrations can do things like query dbt metrics, import definitions, surface the underlying data in partner tools, and more. + +The tools that work with the dbt Semantic Layer include: + +1. **Mode**
+To learn more about integrating with Mode, check out their [documentation](https://mode.com/help/articles/supported-databases/#dbt-semantic-layer) for more info. + +1. **Hex**
+To learn more about integrating with Hex, check out their [documentation](https://learn.hex.tech/docs/connect-to-data/data-connections/dbt-integration#dbt-semantic-layer-integration) for more info. Additionally, refer to [dbt Semantic Layer cells](https://learn.hex.tech/docs/logic-cell-types/transform-cells/dbt-metrics-cells) to set up SQL cells in Hex. + +1. **Google Sheets**
+Google Sheets integration coming soon. + +1. **Tools that allows you to write SQL**
+To connect to tools that allow you to write SQL, they must meet one of the two criteria: + - Supports a generic JDBC driver option (such as DataGrip) or + - Supports Dremio and uses ArrowFlightSQL driver version 12.0.0 or higher. Before you connect to these tools, you'll need to first [set up the dbt Semantic Layer](/docs/use-dbt-semantic-layer/setup-sl) and [generate a service token](/docs/dbt-cloud-apis/service-tokens) to create a Semantic Layer Only and Metadata Only service token. + From a6a1d67ded5a7f89c7cc4385ac47eb9f6128bdf8 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Fri, 13 Oct 2023 10:50:26 +0100 Subject: [PATCH 8/9] Update website/snippets/_sl-partner-links.md Co-authored-by: Devon Fulcher --- website/snippets/_sl-partner-links.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/snippets/_sl-partner-links.md b/website/snippets/_sl-partner-links.md index 2f62974d810..c78479c8111 100644 --- a/website/snippets/_sl-partner-links.md +++ b/website/snippets/_sl-partner-links.md @@ -17,7 +17,7 @@ Google Sheets integration coming soon. 1. **Tools that allows you to write SQL**
To connect to tools that allow you to write SQL, they must meet one of the two criteria: - Supports a generic JDBC driver option (such as DataGrip) or - - Supports Dremio and uses ArrowFlightSQL driver version 12.0.0 or higher. + - Uses Arrow Flight SQL JDBC driver version 12.0.0 or higher. Before you connect to these tools, you'll need to first [set up the dbt Semantic Layer](/docs/use-dbt-semantic-layer/setup-sl) and [generate a service token](/docs/dbt-cloud-apis/service-tokens) to create a Semantic Layer Only and Metadata Only service token. From 12d3e7ecb9979e52a8eca0a9aed2906d621bac5d Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Fri, 13 Oct 2023 11:00:01 +0100 Subject: [PATCH 9/9] Update avail-sl-integrations.md --- .../docs/docs/use-dbt-semantic-layer/avail-sl-integrations.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/website/docs/docs/use-dbt-semantic-layer/avail-sl-integrations.md b/website/docs/docs/use-dbt-semantic-layer/avail-sl-integrations.md index ea5833d586b..aa464aced4a 100644 --- a/website/docs/docs/use-dbt-semantic-layer/avail-sl-integrations.md +++ b/website/docs/docs/use-dbt-semantic-layer/avail-sl-integrations.md @@ -26,11 +26,11 @@ import AvailIntegrations from '/snippets/_sl-partner-links.md'; ## Custom integration -- You can create custom integrations using different languages and tools. We support connecting with JDBC, ADBC, and a GraphQL APIs. For more info, check out [our examples on GitHub](https://github.com/dbt-labs/example-semantic-layer-clients/). +- You can create custom integrations using different languages and tools. We support connecting with JDBC, ADBC, and GraphQL APIs. For more info, check out [our examples on GitHub](https://github.com/dbt-labs/example-semantic-layer-clients/). - You can also connect to tools that allow you to write SQL. These tools must meet one of the two criteria: - Supports a generic JDBC driver option (such as DataGrip) or - - Supports Dremio and uses ArrowFlightSQL driver version 12.0.0 or higher. + - Uses Arrow Flight SQL JDBC driver version 12.0.0 or higher. ## Related docs