From 05f7906776078e0125a8760a3f4acac718b5dcbb Mon Sep 17 00:00:00 2001 From: rpourzand Date: Fri, 11 Aug 2023 14:56:00 -0700 Subject: [PATCH 01/47] Update sl-graphql.md Update with some of Will's graphql content --- .../docs/docs/dbt-cloud-apis/sl-graphql.md | 87 ++++++++++++++++++- 1 file changed, 84 insertions(+), 3 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 43c43b68441..abc13c85cd9 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -26,12 +26,20 @@ The dbt Semantic Layer GraphQL API allows you to explore and query metrics and d dbt Partners can use the Semantic Layer GraphQL API to build and integration with the dbt Semantic Layer. +## Requirements to use the GraphQL API +- A dbt cloud project on 1.6+ +- Metrics are defined and configured +- A dbt Cloud Service Token that has semantic layer access +- Connection to the data warehouse is configured in the project + + ## Using the GraphQL API If you're a dbt user or partner with access to dbt Cloud and the [dbt Semantic Layer](/docs/use-dbt-semantic-layer/dbt-sl), you can [setup](/docs/use-dbt-semantic-layer/setup-sl) and test this API with data from your own instance by configuring the Semantic Layer and obtaining the right GQL connection parameters described in this document. Refer to [Get started with the dbt Semantic Layer](docs/use-dbt-semantic-layer/quickstart-sl) for more info. + ### Authentication Authentication uses a dbt Cloud [service account tokens](/docs/dbt-cloud-apis/service-tokens) passed through a header as follows. To explore the schema, you can enter this information in the "header" section. @@ -40,7 +48,7 @@ Authentication uses a dbt Cloud [service account tokens](/docs/dbt-cloud-apis/se {"Authorization": "Bearer "} ``` -Each GQL request also comes with a dbt Cloud environmentId. The API uses both the service token in the header and environmentId for authentication. +Each GQL request also requires a dbt Cloud `environmentId`. The API uses both the service token in the header and environmentId for authentication. ### Metric metadata calls @@ -88,9 +96,17 @@ metrics: [String!]! dimension: String! ``` -### Metric value query parameters +### Metric Query + +```graphql +query( + environmentId: BigInt! + queryId: String! +): QueryResult! +``` + -The mutation is `createQuery`. The parameters are as follows: +**Create Query** ```graphql createQuery( @@ -105,3 +121,68 @@ order: [String!] = null ): String ``` +**Compile from Jinja** + +```graphql +compileSqlFromJinja( + environmentId: BigIn! + query: String! +): CompileSqlFromJinjaResult! +``` + +**Metric Types** + +```graphql +Metric { + name: String! + description: String + type: MetricType! + typeParams: MetricTypeParams! + filter: WhereFilter + dimensions: [Dimension!]! +} +``` + +``` +MetricType = [SIMPLE, RATIO, CUMULATIVE, DERIVED] +``` + +**Metric Type Parameters** + +```graphql +MetricTypeParams { + measure: MetricInputMeasure + inputMeasures: [MetricInputMeasure!]! + numerator: MetricInput + denominator: MetricInput + expr: String + window: MetricTimeWindow + grainToDate: TimeGranularity + metrics: [MetricInput!] +} +``` + +**Where filter** + +```graphql +WhereFilter { + whereSqlTemplate: String! +} +``` + +**Dimension Types** + +```graphql +Dimension { + name: String! + qualifiedName: String! + description: String + type: DimensionType! + typeParams: DimensionTypeParams + isPartition: Boolean! + expr: String +} +``` +``` +DimensionType = [CATEGORICAL, TIME] +``` From 7de73b81124203c38b09f0a62640ab031a4a6d1d Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Fri, 11 Aug 2023 21:56:08 -0400 Subject: [PATCH 02/47] Update website/docs/docs/dbt-cloud-apis/sl-graphql.md --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index abc13c85cd9..278bd10e78b 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -96,7 +96,7 @@ metrics: [String!]! dimension: String! ``` -### Metric Query +### Metric query ```graphql query( From 8f43b6110c164a61309bfce9b41942334e81b0eb Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Fri, 11 Aug 2023 21:56:30 -0400 Subject: [PATCH 03/47] Update website/docs/docs/dbt-cloud-apis/sl-graphql.md --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 278bd10e78b..eee6dca6f18 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -106,7 +106,7 @@ query( ``` -**Create Query** +**Create query** ```graphql createQuery( From 7a20527967a05d1bdcdc196417b3ce4141254bc6 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Mon, 14 Aug 2023 13:23:51 -0400 Subject: [PATCH 04/47] Update website/docs/docs/dbt-cloud-apis/sl-graphql.md --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index eee6dca6f18..af0470d859c 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -29,7 +29,7 @@ dbt Partners can use the Semantic Layer GraphQL API to build and integration wit ## Requirements to use the GraphQL API - A dbt cloud project on 1.6+ - Metrics are defined and configured -- A dbt Cloud Service Token that has semantic layer access +- A dbt Cloud [service token](/docs/dbt-cloud-apis/service-tokens) that has Semantic Layer access - Connection to the data warehouse is configured in the project From b36f5d3db8b3340cb18b896cf4c894e6d31f5f96 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Mon, 14 Aug 2023 13:24:01 -0400 Subject: [PATCH 05/47] Update website/docs/docs/dbt-cloud-apis/sl-graphql.md --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index af0470d859c..170955d6e1c 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -27,7 +27,7 @@ The dbt Semantic Layer GraphQL API allows you to explore and query metrics and d dbt Partners can use the Semantic Layer GraphQL API to build and integration with the dbt Semantic Layer. ## Requirements to use the GraphQL API -- A dbt cloud project on 1.6+ +- A dbt cloud project on dbt v1.6 or higher - Metrics are defined and configured - A dbt Cloud [service token](/docs/dbt-cloud-apis/service-tokens) that has Semantic Layer access - Connection to the data warehouse is configured in the project From b77481fe9ad7e48511e5d94a0115536af966ff08 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Mon, 14 Aug 2023 13:50:08 -0400 Subject: [PATCH 06/47] Update website/docs/docs/dbt-cloud-apis/sl-graphql.md --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 170955d6e1c..7300cb7d37e 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -30,7 +30,7 @@ dbt Partners can use the Semantic Layer GraphQL API to build and integration wit - A dbt cloud project on dbt v1.6 or higher - Metrics are defined and configured - A dbt Cloud [service token](/docs/dbt-cloud-apis/service-tokens) that has Semantic Layer access -- Connection to the data warehouse is configured in the project +- Your dbt project is configured and connected to a data platform ## Using the GraphQL API From f705ed1b07b74d7c913093f0bd0691844c9e0f15 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Mon, 14 Aug 2023 13:50:15 -0400 Subject: [PATCH 07/47] Update website/docs/docs/dbt-cloud-apis/sl-graphql.md --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 7300cb7d37e..3f24d6bfb18 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -125,7 +125,7 @@ order: [String!] = null ```graphql compileSqlFromJinja( - environmentId: BigIn! + environmentId: BigInt! query: String! ): CompileSqlFromJinjaResult! ``` From dcba2527e00b7d27fa31a18e38ebe6ddd4a67302 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Mon, 14 Aug 2023 14:23:46 -0400 Subject: [PATCH 08/47] Update sl-graphql.md fix code indentions --- .../docs/docs/dbt-cloud-apis/sl-graphql.md | 52 +++++++++---------- 1 file changed, 26 insertions(+), 26 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 3f24d6bfb18..355942ac3b7 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -100,8 +100,8 @@ dimension: String! ```graphql query( - environmentId: BigInt! - queryId: String! +environmentId: BigInt! +queryId: String! ): QueryResult! ``` @@ -125,8 +125,8 @@ order: [String!] = null ```graphql compileSqlFromJinja( - environmentId: BigInt! - query: String! +environmentId: BigInt! +query: String! ): CompileSqlFromJinjaResult! ``` @@ -134,12 +134,12 @@ compileSqlFromJinja( ```graphql Metric { - name: String! - description: String - type: MetricType! - typeParams: MetricTypeParams! - filter: WhereFilter - dimensions: [Dimension!]! + name: String! + description: String + type: MetricType! + typeParams: MetricTypeParams! + filter: WhereFilter + dimensions: [Dimension!]! } ``` @@ -151,14 +151,14 @@ MetricType = [SIMPLE, RATIO, CUMULATIVE, DERIVED] ```graphql MetricTypeParams { - measure: MetricInputMeasure - inputMeasures: [MetricInputMeasure!]! - numerator: MetricInput - denominator: MetricInput - expr: String - window: MetricTimeWindow - grainToDate: TimeGranularity - metrics: [MetricInput!] + measure: MetricInputMeasure + inputMeasures: [MetricInputMeasure!]! + numerator: MetricInput + denominator: MetricInput + expr: String + window: MetricTimeWindow + grainToDate: TimeGranularity + metrics: [MetricInput!] } ``` @@ -166,7 +166,7 @@ MetricTypeParams { ```graphql WhereFilter { - whereSqlTemplate: String! + whereSqlTemplate: String! } ``` @@ -174,13 +174,13 @@ WhereFilter { ```graphql Dimension { - name: String! - qualifiedName: String! - description: String - type: DimensionType! - typeParams: DimensionTypeParams - isPartition: Boolean! - expr: String + name: String! + qualifiedName: String! + description: String + type: DimensionType! + typeParams: DimensionTypeParams + isPartition: Boolean! + expr: String } ``` ``` From 3107fb863591d3a0b0f2d80a4022dd9954608b24 Mon Sep 17 00:00:00 2001 From: rpourzand Date: Fri, 18 Aug 2023 14:52:29 -0700 Subject: [PATCH 09/47] Update sl-graphql.md - Removed start/end time - Made a note about where filters no longer being a string (need to update) - Made a note about adding examples when we're ready - Made a note about the arrow --> Json format so we can add when ready This page is still in progress and shouldn't be published --- .../docs/docs/dbt-cloud-apis/sl-graphql.md | 35 ++++++++----------- 1 file changed, 14 insertions(+), 21 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 355942ac3b7..282fb9cbe93 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -50,9 +50,11 @@ Authentication uses a dbt Cloud [service account tokens](/docs/dbt-cloud-apis/se Each GQL request also requires a dbt Cloud `environmentId`. The API uses both the service token in the header and environmentId for authentication. -### Metric metadata calls +### Output Format + +By default, output is in Arrow format. You can use the following parameter to convert to JSON. Due to performance limitations, we recommend the JSON parameter be only used for testing and validation. (Insert the paramater and an example here) -Use the following example calls to provide you with an idea of the types of commands you can use: +### Metric metadata calls **Fetch available metrics** @@ -96,7 +98,7 @@ metrics: [String!]! dimension: String! ``` -### Metric query +### Metric queries ```graphql query( @@ -114,22 +116,11 @@ environmentId: Int! metrics: [String!]! dimensions: [String!] = null limit: Int = null -startTime: String = null -endTime: String = null -where: String = null +where: [String] = null order: [String!] = null ): String ``` -**Compile from Jinja** - -```graphql -compileSqlFromJinja( -environmentId: BigInt! -query: String! -): CompileSqlFromJinjaResult! -``` - **Metric Types** ```graphql @@ -163,12 +154,7 @@ MetricTypeParams { ``` **Where filter** - -```graphql -WhereFilter { - whereSqlTemplate: String! -} -``` +Update to what's decided **Dimension Types** @@ -186,3 +172,10 @@ Dimension { ``` DimensionType = [CATEGORICAL, TIME] ``` + +### Examples + +(insert examples akin to JDBC once we finalize format) + + + From c69ec92bb255a24fe2b27520b6237592a8142e30 Mon Sep 17 00:00:00 2001 From: rpourzand Date: Mon, 21 Aug 2023 17:41:46 -0700 Subject: [PATCH 10/47] Update sl-graphql.md Adding examples and json updates --- .../docs/docs/dbt-cloud-apis/sl-graphql.md | 124 +++++++++++++++++- 1 file changed, 122 insertions(+), 2 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 282fb9cbe93..c9b16991756 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -52,7 +52,20 @@ Each GQL request also requires a dbt Cloud `environmentId`. The API uses both th ### Output Format -By default, output is in Arrow format. You can use the following parameter to convert to JSON. Due to performance limitations, we recommend the JSON parameter be only used for testing and validation. (Insert the paramater and an example here) +By default, output provided is in Arrow format. You can use the following parameter to convert to JSON. Due to performance limitations, we recommend the JSON parameter be only used for testing and validation. The JSON returned is a base64 encoded string. This means to access the JSON you must run it through a base64 decoder then you should receive the results in JSON. This is a JSON generated from pandas, this means you can either parse this back into a dataframe via pandas.read_json(json, orient="table") or just use the data directly with json["data"] and get the table schema with json["schema"]["fields"] + + +```graphql +{ + query(environmentId: , queryId: ) { + sql + status + error + arrowResult + jsonResult + } +} +``` ### Metric metadata calls @@ -121,6 +134,7 @@ order: [String!] = null ): String ``` + **Metric Types** ```graphql @@ -175,7 +189,113 @@ DimensionType = [CATEGORICAL, TIME] ### Examples -(insert examples akin to JDBC once we finalize format) +**Query two metrics grouped by time** +```graphql +mutation { + createQuery( + environmentId: + metrics: ["food_order_amount", "order_gross_profit"] + groupBy: ["metric_time"] - TODO update with syntax for object + ) { + queryId + } +} +``` +**Query with a time grain** + +```graphql +mutation { + createQuery( + environmentId: + metrics: ["food_order_amount", "order_gross_profit"] + groupBy: ["metric_time__month"] - TODO update with syntax for object + ) { + queryId + } +} +``` + +**Query with a time grain** + +```graphql +mutation { + createQuery( + environmentId: + metrics: ["food_order_amount", "order_gross_profit"] + groupBy: ["metric_time__month"] - TODO update with syntax for object + ) { + queryId + } +} +``` + +**Query with a categorical Dimension** + +```graphql +mutation { + createQuery( + environmentId: + metrics: ["food_order_amount", "order_gross_profit"] + groupBy: ["metric_time__month", "customer__customer_type")] - TODO update with syntax for object + ) { + queryId + } +} +``` + +**Query with a Where Filter** + + The where filter takes a list argument (or a string for a single input). Depending on the object you are filtering on, there are a couple of parameters: + + - `Dimension()` - This is used for any categorical or time dimensions. If used for a time dimension, granularity is required - `Dimension('metric_time').grain('week')` or `Dimension('customer__country')` +- `Entity()` - used for entities like primary and foreign keys - `Entity('order_id')` + +Note: If you prefer more strongly typed `where` clause, you can optionally use `TimeDimension` to separate out categorical dimensions from time ones. The `TimeDimension` input takes the time dimension name and also requires granularity - an example is `TimeDimension('metric_time', 'MONTH')`. + +```graphql +mutation { + createQuery( + metrics:["order_total"] + groupBy:["customer__customer_type"] + where:["{{ Dimension('customer__customer_type') }} = 'new'", "{{ TimeDimension('metric_time', 'MONTH') }} > '2022-10-01'"] - TODO UPDATE THIS + ) { + queryId + } +} +``` + + +**Query with Limit and Order By** + +```graphql +mutation { + createQuery( + environmentId: + metrics: ["food_order_amount", "order_gross_profit"] + groupBy: ["metric_time__month", "customer__customer_type)] - TODO update with syntax for object + limit: TODO + orderBy: TODO + + ) { + queryId + } +} +``` + +**Query with Explain** + +```graphql +mutation { + createQuery( + environmentId: + metrics: ["food_order_amount", "order_gross_profit"] + groupBy: ["metric_time__month", "customer__customer_type)] - TODO update with syntax for object + explain: TODO + ) { + queryId + } +} +``` From 2aaa084ee9e1c55a9574c9dab9f5196652e050c3 Mon Sep 17 00:00:00 2001 From: rpourzand Date: Mon, 21 Aug 2023 17:45:19 -0700 Subject: [PATCH 11/47] Update sl-graphql.md typos --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index c9b16991756..0d4d8c12933 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -238,7 +238,7 @@ mutation { createQuery( environmentId: metrics: ["food_order_amount", "order_gross_profit"] - groupBy: ["metric_time__month", "customer__customer_type")] - TODO update with syntax for object + groupBy: ["metric_time__month", "customer__customer_type"] - TODO update with syntax for object ) { queryId } @@ -274,7 +274,7 @@ mutation { createQuery( environmentId: metrics: ["food_order_amount", "order_gross_profit"] - groupBy: ["metric_time__month", "customer__customer_type)] - TODO update with syntax for object + groupBy: ["metric_time__month", "customer__customer_type")] - TODO update with syntax for object limit: TODO orderBy: TODO @@ -291,7 +291,7 @@ mutation { createQuery( environmentId: metrics: ["food_order_amount", "order_gross_profit"] - groupBy: ["metric_time__month", "customer__customer_type)] - TODO update with syntax for object + groupBy: ["metric_time__month", "customer__customer_type"] - TODO update with syntax for object explain: TODO ) { queryId From ee86b566687d8bfab3ce89d2375f644b9da42d58 Mon Sep 17 00:00:00 2001 From: rpourzand Date: Mon, 21 Aug 2023 17:48:18 -0700 Subject: [PATCH 12/47] Update sl-graphql.md fixing duplicates --- .../docs/docs/dbt-cloud-apis/sl-graphql.md | 20 ++----------------- 1 file changed, 2 insertions(+), 18 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 0d4d8c12933..87d4d2f3312 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -167,8 +167,6 @@ MetricTypeParams { } ``` -**Where filter** -Update to what's decided **Dimension Types** @@ -217,20 +215,6 @@ mutation { } ``` -**Query with a time grain** - -```graphql -mutation { - createQuery( - environmentId: - metrics: ["food_order_amount", "order_gross_profit"] - groupBy: ["metric_time__month"] - TODO update with syntax for object - ) { - queryId - } -} -``` - **Query with a categorical Dimension** ```graphql @@ -247,12 +231,12 @@ mutation { **Query with a Where Filter** - The where filter takes a list argument (or a string for a single input). Depending on the object you are filtering on, there are a couple of parameters: +The where filter takes a list argument (or a string for a single input). Depending on the object you are filtering on, there are a couple of parameters: - `Dimension()` - This is used for any categorical or time dimensions. If used for a time dimension, granularity is required - `Dimension('metric_time').grain('week')` or `Dimension('customer__country')` - `Entity()` - used for entities like primary and foreign keys - `Entity('order_id')` -Note: If you prefer more strongly typed `where` clause, you can optionally use `TimeDimension` to separate out categorical dimensions from time ones. The `TimeDimension` input takes the time dimension name and also requires granularity - an example is `TimeDimension('metric_time', 'MONTH')`. +Note: If you prefer more strongly typed `where` clause, you can optionally use `TimeDimension()` to separate out categorical dimensions from time ones. The `TimeDimension` input takes the time dimension name and also requires granularity - an example is `TimeDimension('metric_time', 'MONTH')`. ```graphql mutation { From 8d78926515003ae755593935eb1a5a21b05c8ec2 Mon Sep 17 00:00:00 2001 From: rpourzand Date: Tue, 22 Aug 2023 18:09:45 -0700 Subject: [PATCH 13/47] Update sl-graphql.md adding limit and order examples --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 87d4d2f3312..553776be3c5 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -259,9 +259,8 @@ mutation { environmentId: metrics: ["food_order_amount", "order_gross_profit"] groupBy: ["metric_time__month", "customer__customer_type")] - TODO update with syntax for object - limit: TODO - orderBy: TODO - + order: [-order_gross_profit] + limit: 10 ) { queryId } From 2cd3b50b26813fbdeae370f06aa239e5d23ad359 Mon Sep 17 00:00:00 2001 From: rpourzand Date: Tue, 22 Aug 2023 18:10:21 -0700 Subject: [PATCH 14/47] Update sl-graphql.md order/limit typo --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 553776be3c5..994f4ad1330 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -259,7 +259,7 @@ mutation { environmentId: metrics: ["food_order_amount", "order_gross_profit"] groupBy: ["metric_time__month", "customer__customer_type")] - TODO update with syntax for object - order: [-order_gross_profit] + order: ["-order_gross_profit"] limit: 10 ) { queryId From 53e1314daa4376e69a4aaa1eebbcfbc7cf3a96b5 Mon Sep 17 00:00:00 2001 From: rpourzand Date: Tue, 22 Aug 2023 18:16:47 -0700 Subject: [PATCH 15/47] Update sl-graphql.md adding data platform call --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 994f4ad1330..4b9f8184d2f 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -67,7 +67,21 @@ By default, output provided is in Arrow format. You can use the following parame } ``` -### Metric metadata calls + + +### Metadata calls + +**Fetch Data Platform Dialect** + +In some cases in your application, it may be useful to know the dialect/data platform that's being used under the hood for the dbt Semantic Layer connection (e.g., if you are building where filters from a GUI rather than user-inputted SQL) The GraphQL API has an easy way to fetch this via the following query: + +```graphql +{ + environmentInfo(environmentId: 100) { + dialect + } +} +``` **Fetch available metrics** From 204ca658d9809249685cd88994ec35e701d50440 Mon Sep 17 00:00:00 2001 From: rpourzand Date: Fri, 25 Aug 2023 15:37:13 -0700 Subject: [PATCH 16/47] Update sl-graphql.md --- .../docs/docs/dbt-cloud-apis/sl-graphql.md | 24 +++++++++---------- 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 4b9f8184d2f..3e5c25f4aeb 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -98,7 +98,7 @@ metrics: [String!]! ): [Dimension!]! ``` -**Fetch available time granularities given metrics** +**Fetch available primary time granularities given metrics** ```graphql queryableGranularities( @@ -207,8 +207,8 @@ DimensionType = [CATEGORICAL, TIME] mutation { createQuery( environmentId: - metrics: ["food_order_amount", "order_gross_profit"] - groupBy: ["metric_time"] - TODO update with syntax for object + metrics: [{name: "food_order_amount"}] + groupBy: [{name: "metric_time}, {name:"customer__customer_type"}] ) { queryId } @@ -221,22 +221,22 @@ mutation { mutation { createQuery( environmentId: - metrics: ["food_order_amount", "order_gross_profit"] - groupBy: ["metric_time__month"] - TODO update with syntax for object + metrics: [{name:"order_total"}] + groupBy: [{name:"metric_time", grain:"month"}] ) { queryId } } ``` -**Query with a categorical Dimension** +**Query two metrics with a categorical Dimension** ```graphql mutation { createQuery( environmentId: - metrics: ["food_order_amount", "order_gross_profit"] - groupBy: ["metric_time__month", "customer__customer_type"] - TODO update with syntax for object + metrics: [{name: "food_order_amount"}, {name: "order_gross_profit"}] + groupBy: [{name: "metric_time, grain: "month"}, {name:"customer__customer_type"}] ) { queryId } @@ -271,8 +271,8 @@ mutation { mutation { createQuery( environmentId: - metrics: ["food_order_amount", "order_gross_profit"] - groupBy: ["metric_time__month", "customer__customer_type")] - TODO update with syntax for object + metrics: [{name: "food_order_amount"}, {name: "order_gross_profit"}] + groupBy: [{name: "metric_time, grain: "month"}, {name:"customer__customer_type"}] order: ["-order_gross_profit"] limit: 10 ) { @@ -287,8 +287,8 @@ mutation { mutation { createQuery( environmentId: - metrics: ["food_order_amount", "order_gross_profit"] - groupBy: ["metric_time__month", "customer__customer_type"] - TODO update with syntax for object + metrics: [{name: "food_order_amount"} {name: "order_gross_profit"}] + groupBy: [{name: "metric_time, grain: "month"}, {name:"customer__customer_type"}] explain: TODO ) { queryId From 80a174f836830865a2f587ecbd84c10eacf232fd Mon Sep 17 00:00:00 2001 From: rpourzand Date: Fri, 25 Aug 2023 15:44:16 -0700 Subject: [PATCH 17/47] Update sl-graphql.md --- .../docs/docs/dbt-cloud-apis/sl-graphql.md | 25 ++++++++++--------- 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 3e5c25f4aeb..674d06df649 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -254,13 +254,14 @@ Note: If you prefer more strongly typed `where` clause, you can optionally use ` ```graphql mutation { - createQuery( - metrics:["order_total"] - groupBy:["customer__customer_type"] - where:["{{ Dimension('customer__customer_type') }} = 'new'", "{{ TimeDimension('metric_time', 'MONTH') }} > '2022-10-01'"] - TODO UPDATE THIS - ) { - queryId - } + createQuery( + environmentId: + metrics:[{name:"order_total"}] + groupBy:[{name:"customer__customer_type"}] + where:["{{ Dimension('customer__customer_type') }} = 'new'", "{{ TimeDimension('metric_time', 'MONTH') }} > '2022-10-01'"] - TODO UPDATE THIS + ) { + queryId + } } ``` @@ -271,8 +272,8 @@ mutation { mutation { createQuery( environmentId: - metrics: [{name: "food_order_amount"}, {name: "order_gross_profit"}] - groupBy: [{name: "metric_time, grain: "month"}, {name:"customer__customer_type"}] + metrics: [{name:"food_order_amount"}, {name: "order_gross_profit"}] + groupBy: [{name:"metric_time, grain: "month"}, {name:"customer__customer_type"}] order: ["-order_gross_profit"] limit: 10 ) { @@ -287,9 +288,9 @@ mutation { mutation { createQuery( environmentId: - metrics: [{name: "food_order_amount"} {name: "order_gross_profit"}] - groupBy: [{name: "metric_time, grain: "month"}, {name:"customer__customer_type"}] - explain: TODO + metrics: [{name:"food_order_amount"} {name:"order_gross_profit"}] + groupBy: [{name:"metric_time, grain:"month"}, {name:"customer__customer_type"}] + explain: TODO ADD ) { queryId } From 1c6d58e15cbf681e67118227f9ed255922a0c78a Mon Sep 17 00:00:00 2001 From: rpourzand Date: Mon, 28 Aug 2023 11:10:52 -0700 Subject: [PATCH 18/47] Update sl-graphql.md will and I went through and added some notes --- .../docs/docs/dbt-cloud-apis/sl-graphql.md | 48 ++++++++++++------- 1 file changed, 32 insertions(+), 16 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 674d06df649..75e73d919d4 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -50,7 +50,9 @@ Authentication uses a dbt Cloud [service account tokens](/docs/dbt-cloud-apis/se Each GQL request also requires a dbt Cloud `environmentId`. The API uses both the service token in the header and environmentId for authentication. -### Output Format +### Output Format & Pagination + +**Output Format** By default, output provided is in Arrow format. You can use the following parameter to convert to JSON. Due to performance limitations, we recommend the JSON parameter be only used for testing and validation. The JSON returned is a base64 encoded string. This means to access the JSON you must run it through a base64 decoder then you should receive the results in JSON. This is a JSON generated from pandas, this means you can either parse this back into a dataframe via pandas.read_json(json, orient="table") or just use the data directly with json["data"] and get the table schema with json["schema"]["fields"] @@ -62,11 +64,17 @@ By default, output provided is in Arrow format. You can use the following parame status error arrowResult - jsonResult + jsonResult (orient: table) } } ``` +The results default to table but you can change it to any [pandas](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_json.html) supported value. + +**Pagination** +By default, we return 1024 rows. If your result set exceeds this, you will have to increment the page by using the following argument: + +TODO:We need to fill this in ### Metadata calls @@ -106,6 +114,7 @@ environmentId: Int! metrics: [String!]! ): [TimeGranularity!]! ``` +TODO: add information on queryable granularities when we make the change. **Fetch available metrics given a set of a dimensions** @@ -119,10 +128,13 @@ dimensions: [String!]! **Fetch dimension values for metrics and a given dimension** ```graphql -dimensionValues( -environmentId: Int! -metrics: [String!]! -dimension: String! +{ + createDimensionValuesquery( + environmentId: + metrics:[String!]! + group_by:[String!] = null + } +} ``` ### Metric queries @@ -141,7 +153,7 @@ queryId: String! createQuery( environmentId: Int! metrics: [String!]! -dimensions: [String!] = null +group_by: [String!] = null limit: Int = null where: [String] = null order: [String!] = null @@ -183,11 +195,11 @@ MetricTypeParams { **Dimension Types** +todo: add info on granularity ```graphql Dimension { name: String! - qualifiedName: String! description: String type: DimensionType! typeParams: DimensionTypeParams @@ -195,6 +207,7 @@ Dimension { expr: String } ``` + ``` DimensionType = [CATEGORICAL, TIME] ``` @@ -208,7 +221,7 @@ mutation { createQuery( environmentId: metrics: [{name: "food_order_amount"}] - groupBy: [{name: "metric_time}, {name:"customer__customer_type"}] + groupBy: [{name: "metric_time}, {name: "customer__customer_type"}] ) { queryId } @@ -221,8 +234,8 @@ mutation { mutation { createQuery( environmentId: - metrics: [{name:"order_total"}] - groupBy: [{name:"metric_time", grain:"month"}] + metrics: [{name: "order_total"}] + groupBy: [{name: "metric_time", grain: "month"}] ) { queryId } @@ -236,7 +249,7 @@ mutation { createQuery( environmentId: metrics: [{name: "food_order_amount"}, {name: "order_gross_profit"}] - groupBy: [{name: "metric_time, grain: "month"}, {name:"customer__customer_type"}] + groupBy: [{name: "metric_time, grain: "month"}, {name: "customer__customer_type"}] ) { queryId } @@ -273,7 +286,7 @@ mutation { createQuery( environmentId: metrics: [{name:"food_order_amount"}, {name: "order_gross_profit"}] - groupBy: [{name:"metric_time, grain: "month"}, {name:"customer__customer_type"}] + groupBy: [{name:"metric_time, grain: "month"}, {name: "customer__customer_type"}] order: ["-order_gross_profit"] limit: 10 ) { @@ -282,17 +295,20 @@ mutation { } ``` +Todo; add example with order by and time dimension granularity change + **Query with Explain** +This takes the same inputs as the `createQuery` mutation. + ```graphql mutation { - createQuery( + compileSql( environmentId: metrics: [{name:"food_order_amount"} {name:"order_gross_profit"}] groupBy: [{name:"metric_time, grain:"month"}, {name:"customer__customer_type"}] - explain: TODO ADD ) { - queryId + sql } } ``` From b2f5a1eaebc4a618ab4c16d3983027860dc9a466 Mon Sep 17 00:00:00 2001 From: rpourzand Date: Mon, 28 Aug 2023 14:09:04 -0700 Subject: [PATCH 19/47] Update sl-graphql.md A couple of more clean ups --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 75e73d919d4..3bf33fae933 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -74,7 +74,7 @@ The results default to table but you can change it to any [pandas](https://panda By default, we return 1024 rows. If your result set exceeds this, you will have to increment the page by using the following argument: -TODO:We need to fill this in +todo: add batching example ### Metadata calls @@ -107,6 +107,7 @@ metrics: [String!]! ``` **Fetch available primary time granularities given metrics** +todo: change queryable granularities once it can apply to all time dimensions ```graphql queryableGranularities( @@ -114,7 +115,7 @@ environmentId: Int! metrics: [String!]! ): [TimeGranularity!]! ``` -TODO: add information on queryable granularities when we make the change. + **Fetch available metrics given a set of a dimensions** @@ -195,7 +196,7 @@ MetricTypeParams { **Dimension Types** -todo: add info on granularity +todo: add info on granularity when we expose it ```graphql Dimension { @@ -269,9 +270,9 @@ Note: If you prefer more strongly typed `where` clause, you can optionally use ` mutation { createQuery( environmentId: - metrics:[{name:"order_total"}] - groupBy:[{name:"customer__customer_type"}] - where:["{{ Dimension('customer__customer_type') }} = 'new'", "{{ TimeDimension('metric_time', 'MONTH') }} > '2022-10-01'"] - TODO UPDATE THIS + metrics:[{name: "order_total"}] + groupBy:[{name: "customer__customer_type"}, {name: "metric_time", grain: "month"}] + where:["{{ Dimension('customer__customer_type') }} = 'new'", "{{ Dimension('metric_time').grain('month') }} > '2022-10-01'"] ) { queryId } @@ -295,7 +296,7 @@ mutation { } ``` -Todo; add example with order by and time dimension granularity change +Todo: add example with order by and time dimension granularity change **Query with Explain** From e381ba39ce543c8e03fab9e961bd5332b5f29893 Mon Sep 17 00:00:00 2001 From: rpourzand Date: Tue, 29 Aug 2023 12:41:35 -0700 Subject: [PATCH 20/47] Update sl-graphql.md Added a couple of notes on queryable grnaularities --- .../docs/docs/dbt-cloud-apis/sl-graphql.md | 35 +++++++++++++++---- 1 file changed, 29 insertions(+), 6 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 3bf33fae933..708fe27b2f7 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -106,8 +106,9 @@ metrics: [String!]! ): [Dimension!]! ``` -**Fetch available primary time granularities given metrics** -todo: change queryable granularities once it can apply to all time dimensions +**Fetch available granularities given metrics** + +Note: This call for `queryableGranularities` returns only queryable granularities for metric time - the primary time dimension across all metrics selected. ```graphql queryableGranularities( @@ -116,6 +117,29 @@ metrics: [String!]! ): [TimeGranularity!]! ``` +You can also get queryable granularities for dimensions using the `dimensions` call: + +```graphql +{ + dimensions(environmentId: , metrics:["order_total"]) { + name + queryableGranularities # --> ["DAY", "WEEK", "MONTH", "QUARTER", "YEAR"] + } +} +``` + +You can also optionally access it from the metrics endpoint: + +```graphql +{ + metrics(environmentId: ) { + name + dimensions { + name + queryableGranularities + } + } +``` **Fetch available metrics given a set of a dimensions** @@ -279,8 +303,10 @@ mutation { } ``` +**Query with Order** +Todo: Add example -**Query with Limit and Order By** +**Query with Limit** ```graphql mutation { @@ -288,7 +314,6 @@ mutation { environmentId: metrics: [{name:"food_order_amount"}, {name: "order_gross_profit"}] groupBy: [{name:"metric_time, grain: "month"}, {name: "customer__customer_type"}] - order: ["-order_gross_profit"] limit: 10 ) { queryId @@ -296,8 +321,6 @@ mutation { } ``` -Todo: add example with order by and time dimension granularity change - **Query with Explain** This takes the same inputs as the `createQuery` mutation. From 63dc51bdf31fa8e6ddfcfc15adbcde0a840f91b7 Mon Sep 17 00:00:00 2001 From: Will Deng Date: Tue, 29 Aug 2023 16:26:06 -0400 Subject: [PATCH 21/47] updated GraphQL docs --- .../docs/docs/dbt-cloud-apis/sl-graphql.md | 137 ++++++++++-------- 1 file changed, 73 insertions(+), 64 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 708fe27b2f7..a5c06649658 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -50,33 +50,6 @@ Authentication uses a dbt Cloud [service account tokens](/docs/dbt-cloud-apis/se Each GQL request also requires a dbt Cloud `environmentId`. The API uses both the service token in the header and environmentId for authentication. -### Output Format & Pagination - -**Output Format** - -By default, output provided is in Arrow format. You can use the following parameter to convert to JSON. Due to performance limitations, we recommend the JSON parameter be only used for testing and validation. The JSON returned is a base64 encoded string. This means to access the JSON you must run it through a base64 decoder then you should receive the results in JSON. This is a JSON generated from pandas, this means you can either parse this back into a dataframe via pandas.read_json(json, orient="table") or just use the data directly with json["data"] and get the table schema with json["schema"]["fields"] - - -```graphql -{ - query(environmentId: , queryId: ) { - sql - status - error - arrowResult - jsonResult (orient: table) - } -} -``` -The results default to table but you can change it to any [pandas](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_json.html) supported value. - -**Pagination** - -By default, we return 1024 rows. If your result set exceeds this, you will have to increment the page by using the following argument: - -todo: add batching example - - ### Metadata calls **Fetch Data Platform Dialect** @@ -85,7 +58,7 @@ In some cases in your application, it may be useful to know the dialect/data pla ```graphql { - environmentInfo(environmentId: 100) { + environmentInfo(environmentId: BigInt!) { dialect } } @@ -94,15 +67,15 @@ In some cases in your application, it may be useful to know the dialect/data pla **Fetch available metrics** ```graphql -metrics(environmentId: Int!): [Metric!]! +metrics(environmentId: BigInt!): [Metric!]! ``` **Fetch available dimensions for metrics** ```graphql dimensions( -environmentId: Int! -metrics: [String!]! + environmentId: BigInt! + metrics: [String!]! ): [Dimension!]! ``` @@ -112,8 +85,8 @@ Note: This call for `queryableGranularities` returns only queryable granularitie ```graphql queryableGranularities( -environmentId: Int! -metrics: [String!]! + environmentId: BigInt! + metrics: [String!]! ): [TimeGranularity!]! ``` @@ -121,7 +94,7 @@ You can also get queryable granularities for dimensions using the `dimensions` c ```graphql { - dimensions(environmentId: , metrics:["order_total"]) { + dimensions(environmentId: BigInt!, metrics:["order_total"]) { name queryableGranularities # --> ["DAY", "WEEK", "MONTH", "QUARTER", "YEAR"] } @@ -132,7 +105,7 @@ You can also optionally access it from the metrics endpoint: ```graphql { - metrics(environmentId: ) { + metrics(environmentId: BigInt!) { name dimensions { name @@ -145,46 +118,43 @@ You can also optionally access it from the metrics endpoint: ```graphql metricsForDimensions( -environmentId: Int! -dimensions: [String!]! + environmentId: BigInt! + dimensions: [String!]! ): [Metric!]! ``` -**Fetch dimension values for metrics and a given dimension** +**Create Dimension Values query** ```graphql -{ - createDimensionValuesquery( - environmentId: - metrics:[String!]! - group_by:[String!] = null - } -} -``` -### Metric queries +mutation createDimensionValuesQuery( + environmentId: BigInt! + metrics: [String!] + groupBy: [String!]! +): CreateDimensionValuesQueryResult! -```graphql -query( -environmentId: BigInt! -queryId: String! -): QueryResult! ``` - -**Create query** +**Create Metric query** ```graphql createQuery( -environmentId: Int! +environmentId: BigInt! metrics: [String!]! group_by: [String!] = null limit: Int = null where: [String] = null order: [String!] = null -): String +): CreateQueryResult ``` +**Fetch query result** +```graphql +query( + environmentId: BigInt! + queryId: String! +): QueryResult! +``` **Metric Types** @@ -196,6 +166,7 @@ Metric { typeParams: MetricTypeParams! filter: WhereFilter dimensions: [Dimension!]! + queryableGranularities: [TimeGranularity!]! } ``` @@ -230,6 +201,7 @@ Dimension { typeParams: DimensionTypeParams isPartition: Boolean! expr: String + queryableGranularities: [TimeGranularity!]! } ``` @@ -237,14 +209,14 @@ Dimension { DimensionType = [CATEGORICAL, TIME] ``` -### Examples +### Creating Query Examples **Query two metrics grouped by time** ```graphql mutation { createQuery( - environmentId: + environmentId: BigInt! metrics: [{name: "food_order_amount"}] groupBy: [{name: "metric_time}, {name: "customer__customer_type"}] ) { @@ -258,7 +230,7 @@ mutation { ```graphql mutation { createQuery( - environmentId: + environmentId: BigInt! metrics: [{name: "order_total"}] groupBy: [{name: "metric_time", grain: "month"}] ) { @@ -272,7 +244,7 @@ mutation { ```graphql mutation { createQuery( - environmentId: + environmentId: BigInt! metrics: [{name: "food_order_amount"}, {name: "order_gross_profit"}] groupBy: [{name: "metric_time, grain: "month"}, {name: "customer__customer_type"}] ) { @@ -293,7 +265,7 @@ Note: If you prefer more strongly typed `where` clause, you can optionally use ` ```graphql mutation { createQuery( - environmentId: + environmentId: BigInt! metrics:[{name: "order_total"}] groupBy:[{name: "customer__customer_type"}, {name: "metric_time", grain: "month"}] where:["{{ Dimension('customer__customer_type') }} = 'new'", "{{ Dimension('metric_time').grain('month') }} > '2022-10-01'"] @@ -304,14 +276,26 @@ mutation { ``` **Query with Order** -Todo: Add example +```graphql +mutation { + createQuery( + environmentId: BigInt! + metrics: [{name: "order_total"}] + groupBy: [{name: "metric_time", grain: "month"}] + orderBy: [{metric: {name: "order_total"}}, {groupBy: {name: "metric_time", grain: "month"}, descending:true}] + ) { + queryId + } +} +``` + **Query with Limit** ```graphql mutation { createQuery( - environmentId: + environmentId: BigInt! metrics: [{name:"food_order_amount"}, {name: "order_gross_profit"}] groupBy: [{name:"metric_time, grain: "month"}, {name: "customer__customer_type"}] limit: 10 @@ -328,7 +312,7 @@ This takes the same inputs as the `createQuery` mutation. ```graphql mutation { compileSql( - environmentId: + environmentId: BigInt! metrics: [{name:"food_order_amount"} {name:"order_gross_profit"}] groupBy: [{name:"metric_time, grain:"month"}, {name:"customer__customer_type"}] ) { @@ -337,3 +321,28 @@ mutation { } ``` +### Output Format & Pagination + +**Output Format** + +By default, output provided is in Arrow format. You can use the following parameter to convert to JSON. Due to performance limitations, we recommend the JSON parameter be only used for testing and validation. The JSON returned is a base64 encoded string. This means to access the JSON you must run it through a base64 decoder then you should receive the results in JSON. This is a JSON generated from pandas, this means you can either parse this back into a dataframe via pandas.read_json(json, orient="table") or just use the data directly with json["data"] and get the table schema with json["schema"]["fields"] + + +```graphql +{ + query(environmentId: BigInt!, queryId: Int!, pageNum: Int { + sql + status + error + totalPages + arrowResult + jsonResult (orient: table) + } +} +``` +The results default to table but you can change it to any [pandas](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_json.html) supported value. + +**Pagination** + +By default, we return 1024 rows per page. If your result set exceeds this, you will have to increment the page by using the pageNum argument + From 1174e9de456c3d3197efb6380faf3aec87ec6c5e Mon Sep 17 00:00:00 2001 From: rpourzand Date: Tue, 29 Aug 2023 13:31:04 -0700 Subject: [PATCH 22/47] Update sl-graphql.md typo --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index a5c06649658..7f78fe2103b 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -90,7 +90,7 @@ queryableGranularities( ): [TimeGranularity!]! ``` -You can also get queryable granularities for dimensions using the `dimensions` call: +You can also get queryable granularities for all other dimensions using the `dimensions` call: ```graphql { From e6482c67df61a97f6b74f0d129897ed2bcee8cd5 Mon Sep 17 00:00:00 2001 From: rpourzand Date: Tue, 29 Aug 2023 13:38:31 -0700 Subject: [PATCH 23/47] Update sl-graphql.md fixing typo will caught --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 7f78fe2103b..c872552e4ff 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -336,7 +336,7 @@ By default, output provided is in Arrow format. You can use the following parame error totalPages arrowResult - jsonResult (orient: table) + jsonResult (orient: TABLE) } } ``` From db6bc730379020fd1ecff81117a38ab37c946c6b Mon Sep 17 00:00:00 2001 From: Courtney Holcomb Date: Tue, 29 Aug 2023 15:11:30 -0700 Subject: [PATCH 24/47] Add example for fetching measures & aggTimeDimension --- .../docs/docs/dbt-cloud-apis/sl-graphql.md | 22 +++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index c872552e4ff..832ee6c0c39 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -112,6 +112,28 @@ You can also optionally access it from the metrics endpoint: queryableGranularities } } +} +``` + +**Fetch measures** +```graphql +{ + measures(metrics: ["order_total"], environmentId: BigInt!) { + name + aggTimeDimension + } +} +``` +`aggTimeDimension` tells you the name of the dimension that maps to `metric_time` for a given measure. You can also query `measures` from the `metrics` endpoint, which allows you to see what dimensions map to `metric_time` for a given metric: +```graphql +{ + metrics(environmentId: BigInt!) { + measures { + name + aggTimeDimension + } + } +} ``` **Fetch available metrics given a set of a dimensions** From e8ea92f768579cd4e68690caf2fb2f0a5b744600 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Thu, 31 Aug 2023 12:53:20 +0100 Subject: [PATCH 25/47] add graphql to other pages --- website/docs/docs/build/sl-getting-started.md | 10 ++--- .../docs/dbt-cloud-apis/sl-api-overview.md | 12 ++---- .../docs/docs/dbt-cloud-apis/sl-graphql.md | 40 ++++++++++++------- .../avail-sl-integrations.md | 6 +-- .../docs/use-dbt-semantic-layer/dbt-sl.md | 2 +- .../use-dbt-semantic-layer/quickstart-sl.md | 8 ++-- .../sl-partner-integration-guide.md | 4 +- website/sidebars.js | 1 + website/snippets/_new-sl-changes.md | 2 +- website/snippets/_new-sl-setup.md | 2 +- website/snippets/_sl-partner-links.md | 2 +- website/snippets/_sl-plan-info.md | 2 +- .../snippets/_sl-test-and-query-metrics.md | 4 +- website/snippets/_v2-sl-prerequisites.md | 4 +- 14 files changed, 53 insertions(+), 46 deletions(-) diff --git a/website/docs/docs/build/sl-getting-started.md b/website/docs/docs/build/sl-getting-started.md index 227cfee20b3..3feba225819 100644 --- a/website/docs/docs/build/sl-getting-started.md +++ b/website/docs/docs/build/sl-getting-started.md @@ -5,7 +5,7 @@ description: "Learn how to create your first semantic model and metric." sidebar_label: Get started with MetricFlow tags: [Metrics, Semantic Layer] meta: - api_name: dbt Semantic Layer API + api_name: dbt Semantic Layer APIs --- import InstallMetricFlow from '/snippets/_sl-install-metricflow.md'; @@ -34,8 +34,8 @@ To experience the power of a universal [dbt Semantic Layer](/docs/use-dbt-semant - Use Snowflake, BigQuery, Databricks, Redshift, or Postgres (CLI only. dbt Cloud support coming soon). - Create a successful run in the environment where you configure the Semantic Layer. - **Note:** Semantic Layer currently supports the Deployment environment for querying. (_development querying experience coming soon_) -- Set up the [Semantic Layer API](/docs/dbt-cloud-apis/sl-api-overview) in the integrated tool to import metric definitions. - - **Note:** dbt Core or Developer accounts can only query data manually using the [MetricFlow CLI](/docs/build/metricflow-cli) and SQL. To dynamically query metrics using external tools, you must have a dbt Cloud [Team or Enterprise](https://www.getdbt.com/pricing/) account with access to the Semantic Layer API.
+- Set up the [Semantic Layer APIs](/docs/dbt-cloud-apis/sl-api-overview) in the integrated tool to import metric definitions. + - **Note:** dbt Core or Developer accounts can only query data manually using the [MetricFlow CLI](/docs/build/metricflow-cli) and SQL. To dynamically query metrics using external tools, you must have a dbt Cloud [Team or Enterprise](https://www.getdbt.com/pricing/) account with access to the Semantic Layer APIs.
- Understand [MetricFlow's](/docs/build/about-metricflow) key concepts, which powers the revamped dbt Semantic Layer. :::tip @@ -80,9 +80,9 @@ import SlSetUp from '/snippets/_new-sl-setup.md'; You can query your metrics in a JDBC-enabled tool or use existing first-class integrations with the dbt Semantic Layer. -Before you begin, you must have a dbt Cloud Team or Enterprise [multi-tenant](/docs/cloud/about-cloud/regions-ip-addresses) deployment, hosted in North America (cloud.getdbt.com login). +Before you begin, you must have a dbt Cloud Team or Enterprise [multi-tenant](/docs/cloud/about-cloud/regions-ip-addresses) deployment, hosted in North America. -- To learn how to use the JDBC API and what tools you can query it with, refer to the {frontMatter.meta.api_name}.
+- To learn how to use the JDBC or GraphQL API and what tools you can query it with, refer to the {frontMatter.meta.api_name}.
* To authenticate, you need to [generate a service token](/docs/dbt-cloud-apis/service-tokens) with Semantic Layer Only and Metadata Only permissions. * Refer to the [SQL query syntax](/docs/dbt-cloud-apis/sl-jdbc#querying-the-api-for-metric-metadata) to query metrics using the API. diff --git a/website/docs/docs/dbt-cloud-apis/sl-api-overview.md b/website/docs/docs/dbt-cloud-apis/sl-api-overview.md index efe54cbd833..42416765904 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-api-overview.md +++ b/website/docs/docs/dbt-cloud-apis/sl-api-overview.md @@ -1,7 +1,7 @@ --- -title: "Semantic Layer API" +title: "Semantic Layer APIs" id: sl-api-overview -description: "Integrate and query using the Semantic Layer API." +description: "Integrate and query metrics and dimensions in downstream tools using the Semantic Layer APIs" tags: [Semantic Layer, API] hide_table_of_contents: true --- @@ -36,9 +36,8 @@ product="dbt Semantic Layer" plan="dbt Cloud Team and Enterprise" instance="hosted in North America" /> -

-
+
- @@ -27,8 +27,8 @@ import AvailIntegrations from '/snippets/_sl-partner-links.md'; ## Related docs -- {frontMatter.meta.api_name} to learn how to integrate with the JDBC to query your metrics in downstream tools. -- [dbt Semantic Layer API query syntax](/docs/dbt-cloud-apis/sl-jdbc#querying-the-api-for-metric-metadata) +- {frontMatter.meta.api_name} to learn how to integrate with JDBC and GraphQL to query your metrics in downstream tools. +- [dbt Semantic Layer APIs query syntax](/docs/dbt-cloud-apis/sl-jdbc#querying-the-api-for-metric-metadata) diff --git a/website/docs/docs/use-dbt-semantic-layer/dbt-sl.md b/website/docs/docs/use-dbt-semantic-layer/dbt-sl.md index c10feded83a..6cfeecddc25 100644 --- a/website/docs/docs/use-dbt-semantic-layer/dbt-sl.md +++ b/website/docs/docs/use-dbt-semantic-layer/dbt-sl.md @@ -59,7 +59,7 @@ instance="hosted in North America" icon="dbt-bit"/> diff --git a/website/docs/docs/use-dbt-semantic-layer/quickstart-sl.md b/website/docs/docs/use-dbt-semantic-layer/quickstart-sl.md index f6c296cfc2a..0e0c7ae086e 100644 --- a/website/docs/docs/use-dbt-semantic-layer/quickstart-sl.md +++ b/website/docs/docs/use-dbt-semantic-layer/quickstart-sl.md @@ -5,7 +5,7 @@ description: "Use this guide to build and define metrics, set up the dbt Semanti sidebar_label: "Get started with the dbt Semantic Layer" tags: [Semantic Layer] meta: - api_name: dbt Semantic Layer API + api_name: dbt Semantic Layer APIs --- @@ -86,12 +86,12 @@ import SlSetUp from '/snippets/_new-sl-setup.md'; You can query your metrics in a JDBC-enabled tool or use existing first-class integrations with the dbt Semantic Layer. -Before you begin, you must have a dbt Cloud Team or Enterprise [multi-tenant](/docs/cloud/about-cloud/regions-ip-addresses) deployment, hosted in North America (cloud.getdbt.com login). +Before you begin, you must have a dbt Cloud Team or Enterprise [multi-tenant](/docs/cloud/about-cloud/regions-ip-addresses) deployment, hosted in North America. -- To learn how to use the JDBC API and what tools you can query it with, refer to the {frontMatter.meta.api_name}.
+- To learn how to use the JDBC or GraphQL API and what tools you can query it with, refer to the {frontMatter.meta.api_name}.
* To authenticate, you need to [generate a service token](/docs/dbt-cloud-apis/service-tokens) with Semantic Layer Only and Metadata Only permissions. - * Refer to the [SQL query syntax](/docs/dbt-cloud-apis/sl-jdbc#querying-the-api-for-metric-metadata) to query metrics using the API. + * Refer to the [SQL query syntax](/docs/dbt-cloud-apis/sl-jdbc#querying-the-api-for-metric-metadata) to query metrics using the APIs. - To learn more about the sophisticated integrations that connect to the dbt Semantic Layer, refer to [Available integrations](/docs/use-dbt-semantic-layer/avail-sl-integrations) for more info. diff --git a/website/docs/guides/dbt-ecosystem/sl-partner-integration-guide.md b/website/docs/guides/dbt-ecosystem/sl-partner-integration-guide.md index decad95a516..708a02f5cf4 100644 --- a/website/docs/guides/dbt-ecosystem/sl-partner-integration-guide.md +++ b/website/docs/guides/dbt-ecosystem/sl-partner-integration-guide.md @@ -21,7 +21,7 @@ This is an evolving guide that is meant to provide recommendations based on our To build a dbt Semantic Layer integration: -- Initially, we recommend building an integration with the [JDBC](/docs/dbt-cloud-apis/sl-jdbc) followed by enhancements of additional features. Refer to the dedicated [dbt Semantic Layer API](/docs/dbt-cloud-apis/sl-api-overview) for more technical integration details. +- Initially, we recommend building an integration with the [JDBC](/docs/dbt-cloud-apis/sl-jdbc) followed by enhancements of additional features. Refer to the dedicated [dbt Semantic Layer APIs](/docs/dbt-cloud-apis/sl-api-overview) for more technical integration details. - Familiarize yourself with the [dbt Semantic Layer](/docs/use-dbt-semantic-layer/dbt-sl) and [MetricFlow](/docs/build/about-metricflow)'s key concepts. There are two main objects: @@ -122,7 +122,7 @@ For transparency and additional context, we recommend you have an easy way for t These are recommendations on how to evolve a Semantic Layer integration and not a strict runbook. **Stage 1 - The basic** -* Supporting and using the new [JDBC](/docs/dbt-cloud-apis/sl-jdbc) is the first step. Refer to the [dbt Semantic Layer API](/docs/dbt-cloud-apis/sl-api-overview) for more technical details. +* Supporting and using the new [JDBC](/docs/dbt-cloud-apis/sl-jdbc) is the first step. Refer to the [dbt Semantic Layer APIs](/docs/dbt-cloud-apis/sl-api-overview) for more technical details. **Stage 2 - More discoverability and basic querying** * Support listing metrics defined in project diff --git a/website/sidebars.js b/website/sidebars.js index d1a6c4664e7..2a33ef75ff8 100644 --- a/website/sidebars.js +++ b/website/sidebars.js @@ -513,6 +513,7 @@ const sidebarSettings = { link: { type: "doc", id: "docs/dbt-cloud-apis/sl-api-overview" }, items: [ "docs/dbt-cloud-apis/sl-jdbc", + "docs/dbt-cloud-apis/sl-graphql", "docs/dbt-cloud-apis/sl-manifest", ], }, diff --git a/website/snippets/_new-sl-changes.md b/website/snippets/_new-sl-changes.md index fa7c7abf743..6eca327001a 100644 --- a/website/snippets/_new-sl-changes.md +++ b/website/snippets/_new-sl-changes.md @@ -3,6 +3,6 @@ The dbt Semantic Layer has been re-released with [significant improvements](https://www.getdbt.com/blog/dbt-semantic-layer-whats-next/), making it more efficient to define and query metrics. -The new version is available in [public beta](/docs/dbt-versions/release-notes/Aug-2023/sl-revamp-beta#public-beta) and introduces [MetricFlow](/docs/build/about-metricflow), an essential component. It also includes new semantic elements, better governance, improved efficiency, easier data access, and new Semantic Layer API. +The new version is available in [public beta](/docs/dbt-versions/release-notes/Aug-2023/sl-revamp-beta#public-beta) and introduces [MetricFlow](/docs/build/about-metricflow), an essential component. It also includes new semantic elements, better governance, improved efficiency, easier data access, and new dbt Semantic Layer APIs. ::: diff --git a/website/snippets/_new-sl-setup.md b/website/snippets/_new-sl-setup.md index 9f1fcef0fb6..b802db9c5ae 100644 --- a/website/snippets/_new-sl-setup.md +++ b/website/snippets/_new-sl-setup.md @@ -25,7 +25,7 @@ If you're using the legacy Semantic Layer, we **highly** recommend you [upgrade 5. Select the deployment environment you want for the Semantic Layer and click **Save**. -6. After saving it, you'll be provided with the connection information that allows you to connect to downstream tools. If your tool supports JDBC, save the JDBC URL or individual components (like environment id and host). +6. After saving it, you'll be provided with the connection information that allows you to connect to downstream tools. If your tool supports JDBC, save the JDBC URL or individual components (like environment id and host). If it uses the GraphQL API, save the GraphQL API host information instead. diff --git a/website/snippets/_sl-partner-links.md b/website/snippets/_sl-partner-links.md index 3e4173a6ae3..a37c374f5d5 100644 --- a/website/snippets/_sl-partner-links.md +++ b/website/snippets/_sl-partner-links.md @@ -1,4 +1,4 @@ - + The dbt Semantic Layer integrations are capable of querying dbt metrics, importing definitions, surfacing the underlying data in partner tools, and more. These are the following tools that integrate with the dbt Semantic Layer: 1. **Mode** — Refer to the [Mode docs](https://mode.com/help/articles/supported-databases/#dbt-semantic-layer) for info on how to integrate with Mode. diff --git a/website/snippets/_sl-plan-info.md b/website/snippets/_sl-plan-info.md index 5294dab39d8..44a37fa7f9b 100644 --- a/website/snippets/_sl-plan-info.md +++ b/website/snippets/_sl-plan-info.md @@ -1 +1 @@ -During {props.cycle}, you can access {props.product} on {props.plan} multi-tenant plans {props.instance} (cloud.getdbt.com login). It's available on dbt v1.6 or higher. dbt Cloud Developer plans and dbt Core users can use MetricFlow to define and test metrics locally, but can't dynamically query them with integrated tools.

+During {props.cycle}, you can access {props.product} on {props.plan} multi-tenant plans {props.instance}. It's available on dbt v1.6 or higher. dbt Cloud Developer plans and dbt Core users can use MetricFlow to define and test metrics locally, but can't dynamically query them with integrated tools.

diff --git a/website/snippets/_sl-test-and-query-metrics.md b/website/snippets/_sl-test-and-query-metrics.md index 0ccea4dc3e8..388a3452dbe 100644 --- a/website/snippets/_sl-test-and-query-metrics.md +++ b/website/snippets/_sl-test-and-query-metrics.md @@ -2,7 +2,7 @@ Support for testing or querying metrics in the dbt Cloud IDE is not available in the current beta but is coming soon. -You can use the **Preview** or **Compile** buttons in the IDE to run semantic validations and make sure your metrics are defined. You can [dynamically query metrics](#connect-and-query-api) with integrated tools on a dbt Cloud [Team or Enterprise](https://www.getdbt.com/pricing/) plan using the [Semantic Layer API](/docs/dbt-cloud-apis/sl-api-overview). +You can use the **Preview** or **Compile** buttons in the IDE to run semantic validations and make sure your metrics are defined. You can [dynamically query metrics](#connect-and-query-api) with integrated tools on a dbt Cloud [Team or Enterprise](https://www.getdbt.com/pricing/) plan using the [dbt Semantic Layer APIs](/docs/dbt-cloud-apis/sl-api-overview). Currently, you can test metrics locally using the MetricFlow CLI. dbt Cloud IDE support is coming soon. Alternatively, you can test using SQL client tools like DataGrip, DBeaver, or RazorSQL. @@ -28,4 +28,4 @@ MetricFlow needs a `semantic_manifest.json` in order to build a semantic graph. 5. Run `mf validate-configs` to run validation on your semantic models and metrics. 6. Commit and merge the code changes that contain the metric definitions. -To streamline your metric querying process, you can connect to the [dbt Semantic Layer API](/docs/dbt-cloud-apis/sl-api-overview) to access your metrics programmatically. For SQL syntax, refer to [Querying the API for metric metadata](/docs/dbt-cloud-apis/sl-jdbc#querying-the-api-for-metric-metadata) to query metrics using the API. +To streamline your metric querying process, you can connect to the [dbt Semantic Layer APIs](/docs/dbt-cloud-apis/sl-api-overview) to access your metrics programmatically. For SQL syntax, refer to [Querying the API for metric metadata](/docs/dbt-cloud-apis/sl-jdbc#querying-the-api-for-metric-metadata) to query metrics using the API. diff --git a/website/snippets/_v2-sl-prerequisites.md b/website/snippets/_v2-sl-prerequisites.md index 632e2af6412..ac73d67a08d 100644 --- a/website/snippets/_v2-sl-prerequisites.md +++ b/website/snippets/_v2-sl-prerequisites.md @@ -8,8 +8,8 @@ To use the Semantic Layer, you must: - Use Snowflake, BigQuery, Databricks, or Redshift (dbt Cloud Postgres support coming soon). - Create a successful run in the environment where you configure the Semantic Layer. - **Note:** Semantic Layer currently supports the Deployment environment for querying. (_development querying experience coming soon_) -- Set up the [Semantic Layer API](/docs/dbt-cloud-apis/sl-api-overview) in the integrated tool to import metric definitions. - - **Note:** dbt Core or Developer accounts can only query data manually using the [MetricFlow CLI](/docs/build/metricflow-cli) and SQL. To dynamically query metrics using external tools, you must have a dbt Cloud [Team or Enterprise](https://www.getdbt.com/pricing/) account with access to the Semantic Layer API.
+- Set up the [dbt Semantic Layer APIs](/docs/dbt-cloud-apis/sl-api-overview) in the integrated tool to import metric definitions. + - **Note:** dbt Core or Developer accounts can only query data manually using the [MetricFlow CLI](/docs/build/metricflow-cli) and SQL. To dynamically query metrics using external tools, you must have a dbt Cloud [Team or Enterprise](https://www.getdbt.com/pricing/) account with access to the dbt Semantic Layer APIs.
- Understand [MetricFlow's](/docs/build/about-metricflow) key concepts, which powers the revamped dbt Semantic Layer. From d5ce90240b483496040c31889b3ad1658ed3f332 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Mon, 4 Sep 2023 11:01:33 +0100 Subject: [PATCH 26/47] Update sl-graphql.md remove 'todo' and fix json typo --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 51fae8923b8..895d3eb1f8b 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -141,7 +141,7 @@ You can also optionally access it from the metrics endpoint: } ``` -**Fetch available metrics given a set of a dimensions** +**Fetch available metrics given a set of dimensions** ```graphql metricsForDimensions( @@ -219,7 +219,6 @@ MetricTypeParams { **Dimension Types** -todo: add info on granularity when we expose it ```graphql Dimension { @@ -291,7 +290,7 @@ The `where` filter takes a list argument (or a string for a single input). Depen - `Entity()` — Used for entities like primary and foreign keys, such as `Entity('order_id')`. -Note: If you prefer more strongly typed `where` clause, you can optionally use `TimeDimension()` to separate out categorical dimensions from time ones. The `TimeDimension` input takes the time dimension name and also requires granularity. For example, `TimeDimension('metric_time', 'MONTH')`. +Note: If you prefer a more strongly typed `where` clause, you can optionally use `TimeDimension()` to separate out categorical dimensions from time ones. The `TimeDimension` input takes the time dimension name and also requires granularity. For example, `TimeDimension('metric_time', 'MONTH')`. ```graphql mutation { @@ -357,7 +356,7 @@ mutation { **Output format** -By default, the output is in Arrow format. You can switch to JASON format using the following parameter. However, due to performances limitations, we recommend using the JSON parameter for testing and validation. The JSON received is a base64 encoded string. To access it, you can decode it using a base64 decoder. The JSON is created from pandas, which means you can change it back to a dataframe using `pandas.read_json(json, orient="table")`. Or you can work with the data directly using `json["data"]`, and find the table schema using `json["schema"]["fields"]`. +By default, the output is in Arrow format. You can switch to JSON format using the following parameter. However, due to performance limitations, we recommend using the JSON parameter for testing and validation. The JSON received is a base64 encoded string. To access it, you can decode it using a base64 decoder. The JSON is created from pandas, which means you can change it back to a dataframe using `pandas.read_json(json, orient="table")`. Or you can work with the data directly using `json["data"]`, and find the table schema using `json["schema"]["fields"]`. ```graphql @@ -373,7 +372,7 @@ By default, the output is in Arrow format. You can switch to JASON format using } ``` -The results default to table but you can change it to any [pandas](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_json.html) supported value. +The results default to the table but you can change it to any [pandas](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_json.html) supported value. **Pagination** From b4a548e7f2135cc9807ee28468677378e48e3444 Mon Sep 17 00:00:00 2001 From: Will Deng Date: Mon, 11 Sep 2023 14:43:33 -0400 Subject: [PATCH 27/47] updated with new changes --- .../docs/docs/dbt-cloud-apis/sl-graphql.md | 61 +++++++++++++------ 1 file changed, 41 insertions(+), 20 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 895d3eb1f8b..64b30fcc169 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -60,9 +60,9 @@ The GraphQL API has an easy way to fetch this with the following query: ```graphql { - environmentInfo(environmentId: BigInt!) { - dialect - } + environmentInfo(environmentId: BigInt!) { + dialect + } } ``` @@ -77,7 +77,7 @@ metrics(environmentId: BigInt!): [Metric!]! ```graphql dimensions( environmentId: BigInt! - metrics: [String!]! + metrics: [MetricInput!]! ): [Dimension!]! ``` @@ -88,7 +88,7 @@ Note: This call for `queryableGranularities` returns only queryable granularitie ```graphql queryableGranularities( environmentId: BigInt! - metrics: [String!]! + metrics: [MetricInput!]! ): [TimeGranularity!]! ``` @@ -96,7 +96,7 @@ You can also get queryable granularities for all other dimensions using the `dim ```graphql { - dimensions(environmentId: BigInt!, metrics:["order_total"]) { + dimensions(environmentId: BigInt!, metrics:[{name:"order_total"}]) { name queryableGranularities # --> ["DAY", "WEEK", "MONTH", "QUARTER", "YEAR"] } @@ -121,7 +121,7 @@ You can also optionally access it from the metrics endpoint: ```graphql { - measures(metrics: ["order_total"], environmentId: BigInt!) { + measures(environmentId: BigInt!, metrics: [{name:"order_total"}]) { name aggTimeDimension } @@ -146,7 +146,7 @@ You can also optionally access it from the metrics endpoint: ```graphql metricsForDimensions( environmentId: BigInt! - dimensions: [String!]! + dimensions: [GroupByInput!]! ): [Metric!]! ``` @@ -156,8 +156,8 @@ metricsForDimensions( mutation createDimensionValuesQuery( environmentId: BigInt! - metrics: [String!] - groupBy: [String!]! + metrics: [MetricInput!] + groupBy: [GroupByInput!]! ): CreateDimensionValuesQueryResult! ``` @@ -166,15 +166,36 @@ mutation createDimensionValuesQuery( ```graphql createQuery( -environmentId: BigInt! -metrics: [String!]! -group_by: [String!] = null -limit: Int = null -where: [String] = null -order: [String!] = null + environmentId: BigInt! + metrics: [MetricInput!]! + groupBy: [GroupByInput!] = null + limit: Int = null + where: [WhereInput!] = null + order: [OrderByInput!] = null ): CreateQueryResult ``` +```graphql +MetricInput { + name: String! +} + +GroupByInput { + name: String! + grain: TimeGranularity = null +} + +WhereInput { + sql: String! +} + +OrderByinput { # -- pass one and only one of metric or groupBy + metric: MetricInput = null + groupBy: GroupByInput = null + descending: Boolean! = false +} +``` + **Fetch query result** ```graphql @@ -298,7 +319,7 @@ mutation { environmentId: BigInt! metrics:[{name: "order_total"}] groupBy:[{name: "customer__customer_type"}, {name: "metric_time", grain: "month"}] - where:["{{ Dimension('customer__customer_type') }} = 'new'", "{{ Dimension('metric_time').grain('month') }} > '2022-10-01'"] + where:[{sql: "{{ Dimension('customer__customer_type') }} = 'new'"}, {sql:"{{ Dimension('metric_time').grain('month') }} > '2022-10-01'"}] ) { queryId } @@ -356,18 +377,18 @@ mutation { **Output format** -By default, the output is in Arrow format. You can switch to JSON format using the following parameter. However, due to performance limitations, we recommend using the JSON parameter for testing and validation. The JSON received is a base64 encoded string. To access it, you can decode it using a base64 decoder. The JSON is created from pandas, which means you can change it back to a dataframe using `pandas.read_json(json, orient="table")`. Or you can work with the data directly using `json["data"]`, and find the table schema using `json["schema"]["fields"]`. +By default, the output is in Arrow format. You can switch to JSON format using the following parameter. However, due to performance limitations, we recommend using the JSON parameter for testing and validation. The JSON received is a base64 encoded string. To access it, you can decode it using a base64 decoder. The JSON is created from pandas, which means you can change it back to a dataframe using `pandas.read_json(json, orient="table")`. Or you can work with the data directly using `json["data"]`, and find the table schema using `json["schema"]["fields"]`. Alternatively, you can pass `encoded:false` to the jsonResult field to get a raw JSON string directly. ```graphql { - query(environmentId: BigInt!, queryId: Int!, pageNum: Int { + query(environmentId: BigInt!, queryId: Int!, pageNum: Int! = 1) { sql status error totalPages arrowResult - jsonResult (orient: TABLE) + jsonResult(orient: PandasJsonOrient! = TABLE, encoded: Boolean! = true) } } ``` From 254ef8c0d6b1552ff5a6531bdbef3136e71d6fe7 Mon Sep 17 00:00:00 2001 From: rpourzand Date: Mon, 18 Sep 2023 15:04:33 -0700 Subject: [PATCH 28/47] Update sl-graphql.md adding the note about granularity output --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 64b30fcc169..bb8734c4ffd 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -289,6 +289,8 @@ mutation { } ``` +Note that when using granularity in the query, the output of a time dimension with a time grain applied to it always takes form of dimension name appended with a double underscore and the granularity level - `{time_dimension_name}__{DAY|WEEK|MONTH|QUARTER|YEAR}`. Even if no granularity is specified, it will also always have a granularity appended to it and will default to the lowest available (usually daily for most data sources). + **Query two metrics with a categorical dimension** ```graphql From eea3ce7fceff783d7fa123c45f40d25a0016b95a Mon Sep 17 00:00:00 2001 From: rpourzand Date: Mon, 18 Sep 2023 15:30:07 -0700 Subject: [PATCH 29/47] Update sl-graphql.md adding a recommended sentence from @WilliamDee --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index bb8734c4ffd..05b3e100a29 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -289,7 +289,7 @@ mutation { } ``` -Note that when using granularity in the query, the output of a time dimension with a time grain applied to it always takes form of dimension name appended with a double underscore and the granularity level - `{time_dimension_name}__{DAY|WEEK|MONTH|QUARTER|YEAR}`. Even if no granularity is specified, it will also always have a granularity appended to it and will default to the lowest available (usually daily for most data sources). +Note that when using granularity in the query, the output of a time dimension with a time grain applied to it always takes form of dimension name appended with a double underscore and the granularity level - `{time_dimension_name}__{DAY|WEEK|MONTH|QUARTER|YEAR}`. Even if no granularity is specified, it will also always have a granularity appended to it and will default to the lowest available (usually daily for most data sources). It is encouraged to specify a granularity when using time dimensions so that there won't be any unexpected results with the output data. **Query two metrics with a categorical dimension** From 1df31d24fcc8fd84ca1591dcff2a09bb3ebd0808 Mon Sep 17 00:00:00 2001 From: rpourzand Date: Tue, 26 Sep 2023 09:43:26 -0700 Subject: [PATCH 30/47] Update sl-graphql.md @aiguofer Can you review this change I made in the docs to update the host for the explorer? I assume the schema explorer host will change once we do this. --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 05b3e100a29..3071d4ddf60 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -22,7 +22,7 @@ GraphQL has several advantages, such as self-documenting, having a strong typing ## dbt Semantic Layer GraphQL API -The dbt Semantic Layer GraphQL API allows you to explore and query metrics and dimensions. Due to it's self-documenting nature, you can explore the calls conveniently through the [schema explorer](https://cloud.getdbt.com/semantic-layer/api/graphql). +The dbt Semantic Layer GraphQL API allows you to explore and query metrics and dimensions. Due to it's self-documenting nature, you can explore the calls conveniently through the [schema explorer](https://semantic-layer.cloud.getdbt.com/api/graphql). dbt Partners can use the Semantic Layer GraphQL API to build and integration with the dbt Semantic Layer. From dd3709722480ee8f81c076113ebb3a70eaa9ebc6 Mon Sep 17 00:00:00 2001 From: Will Deng Date: Thu, 28 Sep 2023 14:28:43 -0400 Subject: [PATCH 31/47] update to add python example for querying --- .../docs/docs/dbt-cloud-apis/sl-graphql.md | 102 ++++++++++++++---- 1 file changed, 81 insertions(+), 21 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 3071d4ddf60..b2604c3e2ca 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -209,12 +209,12 @@ query( ```graphql Metric { - name: String! - description: String - type: MetricType! - typeParams: MetricTypeParams! - filter: WhereFilter - dimensions: [Dimension!]! + name: String! + description: String + type: MetricType! + typeParams: MetricTypeParams! + filter: WhereFilter + dimensions: [Dimension!]! queryableGranularities: [TimeGranularity!]! } ``` @@ -227,14 +227,14 @@ MetricType = [SIMPLE, RATIO, CUMULATIVE, DERIVED] ```graphql MetricTypeParams { - measure: MetricInputMeasure - inputMeasures: [MetricInputMeasure!]! - numerator: MetricInput - denominator: MetricInput - expr: String - window: MetricTimeWindow - grainToDate: TimeGranularity - metrics: [MetricInput!] + measure: MetricInputMeasure + inputMeasures: [MetricInputMeasure!]! + numerator: MetricInput + denominator: MetricInput + expr: String + window: MetricTimeWindow + grainToDate: TimeGranularity + metrics: [MetricInput!] } ``` @@ -243,12 +243,12 @@ MetricTypeParams { ```graphql Dimension { - name: String! - description: String - type: DimensionType! - typeParams: DimensionTypeParams - isPartition: Boolean! - expr: String + name: String! + description: String + type: DimensionType! + typeParams: DimensionTypeParams + isPartition: Boolean! + expr: String queryableGranularities: [TimeGranularity!]! } ``` @@ -352,7 +352,7 @@ mutation { environmentId: BigInt! metrics: [{name:"food_order_amount"}, {name: "order_gross_profit"}] groupBy: [{name:"metric_time, grain: "month"}, {name: "customer__customer_type"}] - limit: 10 + limit: 10 ) { queryId } @@ -400,3 +400,63 @@ The results default to the table but you can change it to any [pandas](https://p **Pagination** By default, we return 1024 rows per page. If your result set exceeds this, you need to increase the page number using the `pageNum` option. + +### Execute a Query through Python + +The `arrowResult` from the GraphQL query response isn't all that useful visually as it's a byte dump. You can utilize any language that supports Arrow to convert that byte into a Arrow table. Here is an example of querying and decoding the arrow result in Python. + +```python +import base64 +import pyarrow as pa + +headers = {"Authorization":"Bearer "} +query_result_request = """ +{ + query(environmentId: 70, queryId: "12345678") { + sql + status + error + arrowResult + } +} +""" + +gql_response = requests.post( + "http://localhost:8000/graphql", + json={"query": query_result_request}, + headers=headers, +) + +""" +gql_response.json() => +{ + "data": { + "query": { + "sql": "SELECT\n ordered_at AS metric_time__day\n , SUM(order_total) AS order_total\nFROM semantic_layer.orders orders_src_1\nGROUP BY\n ordered_at", + "status": "SUCCESSFUL", + "error": null, + "arrowResult": "arrow-byte-data" + } + } +} +""" + +def to_arrow_table(byte_string: str) -> pa.Table: + """Get a raw base64 string and convert to an Arrow Table.""" + with pa.ipc.open_stream(base64.b64decode(res)) as reader: + return pa.Table.from_batches(reader, reader.schema) + + +arrow_table = to_arrow_table(gql_response.json()["data"]["query"]["arrowResult"]) + +# Perform whatever functionality is available e.g., +# Convert to a pandas table +print(arrow_table.to_pandas()) +""" +order_total ordered_at + 3 2023-08-07 + 112 2023-08-08 + 12 2023-08-09 + 5123 2023-08-10 +""" +``` \ No newline at end of file From 50606b6a7ec98868486ca280df3b026c81d4f4c5 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Fri, 29 Sep 2023 16:47:24 +0100 Subject: [PATCH 32/47] Update website/docs/docs/dbt-cloud-apis/sl-graphql.md --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 1 - 1 file changed, 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index b2604c3e2ca..289e85bd71e 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -450,7 +450,6 @@ def to_arrow_table(byte_string: str) -> pa.Table: arrow_table = to_arrow_table(gql_response.json()["data"]["query"]["arrowResult"]) # Perform whatever functionality is available e.g., -# Convert to a pandas table print(arrow_table.to_pandas()) """ order_total ordered_at From ce68d8109f333658223c85d16f53311832419009 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Fri, 29 Sep 2023 16:47:31 +0100 Subject: [PATCH 33/47] Update website/docs/docs/dbt-cloud-apis/sl-graphql.md --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 289e85bd71e..bdae855ef62 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -403,6 +403,8 @@ By default, we return 1024 rows per page. If your result set exceeds this, you n ### Execute a Query through Python +The `arrowResult` in the GraphQL query response is a byte dump, which isn't visually useful. You can convert this byte data into an Arrow table using any Arrow-supported language. Refer to the following Python example explaining how to query and decode the arrow result: + The `arrowResult` from the GraphQL query response isn't all that useful visually as it's a byte dump. You can utilize any language that supports Arrow to convert that byte into a Arrow table. Here is an example of querying and decoding the arrow result in Python. ```python From de1226fa111fc432d9b8684fa8d2543f8f41e535 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Fri, 29 Sep 2023 16:47:39 +0100 Subject: [PATCH 34/47] Update website/docs/docs/dbt-cloud-apis/sl-graphql.md --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index bdae855ef62..df487e3071a 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -451,7 +451,7 @@ def to_arrow_table(byte_string: str) -> pa.Table: arrow_table = to_arrow_table(gql_response.json()["data"]["query"]["arrowResult"]) -# Perform whatever functionality is available e.g., +# Perform whatever functionality is available, like convert to a pandas table. print(arrow_table.to_pandas()) """ order_total ordered_at From 6b7b3f18daa1e53209d44afea8f0905160421957 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Fri, 29 Sep 2023 16:47:47 +0100 Subject: [PATCH 35/47] Update website/docs/docs/dbt-cloud-apis/sl-graphql.md --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index df487e3071a..c53c3374077 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -401,7 +401,7 @@ The results default to the table but you can change it to any [pandas](https://p By default, we return 1024 rows per page. If your result set exceeds this, you need to increase the page number using the `pageNum` option. -### Execute a Query through Python +### Run a Python query The `arrowResult` in the GraphQL query response is a byte dump, which isn't visually useful. You can convert this byte data into an Arrow table using any Arrow-supported language. Refer to the following Python example explaining how to query and decode the arrow result: From e670912b5edb8eae2328d77d78446423913739f5 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Fri, 29 Sep 2023 16:49:17 +0100 Subject: [PATCH 36/47] Update website/docs/docs/dbt-cloud-apis/sl-graphql.md --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 1 - 1 file changed, 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index c53c3374077..d62e16f1334 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -405,7 +405,6 @@ By default, we return 1024 rows per page. If your result set exceeds this, you n The `arrowResult` in the GraphQL query response is a byte dump, which isn't visually useful. You can convert this byte data into an Arrow table using any Arrow-supported language. Refer to the following Python example explaining how to query and decode the arrow result: -The `arrowResult` from the GraphQL query response isn't all that useful visually as it's a byte dump. You can utilize any language that supports Arrow to convert that byte into a Arrow table. Here is an example of querying and decoding the arrow result in Python. ```python import base64 From df5e1cafd38869151e85422509106cfe80eae9d5 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Fri, 29 Sep 2023 16:51:13 +0100 Subject: [PATCH 37/47] Update website/docs/guides/dbt-ecosystem/sl-partner-integration-guide.md --- .../docs/guides/dbt-ecosystem/sl-partner-integration-guide.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/dbt-ecosystem/sl-partner-integration-guide.md b/website/docs/guides/dbt-ecosystem/sl-partner-integration-guide.md index 6fba399512d..b0718a6484d 100644 --- a/website/docs/guides/dbt-ecosystem/sl-partner-integration-guide.md +++ b/website/docs/guides/dbt-ecosystem/sl-partner-integration-guide.md @@ -114,7 +114,7 @@ For better analysis, it's best to have the context of the metrics close to where These are recommendations on how to evolve a Semantic Layer integration and not a strict runbook. **Stage 1 - The basic** -* Supporting and using the new [JDBC](/docs/dbt-cloud-apis/sl-jdbc) is the first step. Refer to the [dbt Semantic Layer APIs](/docs/dbt-cloud-apis/sl-api-overview) for more technical details. +* Supporting and using the new [JDBC](/docs/dbt-cloud-apis/sl-jdbc) or [GraphQL](/docs/dbt-cloud-apis/sl-graphql) is the first step. Refer to the [dbt Semantic Layer APIs](/docs/dbt-cloud-apis/sl-api-overview) for more technical details. **Stage 2 - More discoverability and basic querying** * Support listing metrics defined in the project From 214ec5d16113a60ee552f6c81d4ac1f8281167f6 Mon Sep 17 00:00:00 2001 From: rpourzand Date: Fri, 29 Sep 2023 13:39:50 -0700 Subject: [PATCH 38/47] jdbc clarification We were getting some questions about the generic driver piece so I decided to add it to the jdbc doc itself. It already exists in available integrations. Additionally, I made one change to the available integrations page. --- website/docs/docs/dbt-cloud-apis/sl-jdbc.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md index c238dcad680..02d26229794 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md +++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md @@ -30,6 +30,8 @@ dbt Labs partners can use the JDBC API to build integrations in their tools with If you are a dbt user or partner with access to dbt Cloud and the [dbt Semantic Layer](/docs/use-dbt-semantic-layer/dbt-sl), you can [setup](/docs/use-dbt-semantic-layer/setup-sl) and test this API with data from your own instance by configuring the Semantic Layer and obtaining the right JDBC connection parameters described in this document. +You *may* be able to use our JDBC API with tools that do not have an official integration with the dbt Semantic Layer. If the tool you use allows you to write SQL and either supports a generic JDBC driver option (such as DataGrip) or supports Dremio and uses ArrowFlightSQL driver version 12.0.0 or higher, you can access the Semantic Layer API. + Refer to [Get started with the dbt Semantic Layer](/docs/use-dbt-semantic-layer/quickstart-sl) for more info. ## Authentication From 3a079c2cbcf6eecb71eb06c0df36f89950e316d2 Mon Sep 17 00:00:00 2001 From: Doug Beatty <44704949+dbeatty10@users.noreply.github.com> Date: Fri, 29 Sep 2023 18:08:06 -0600 Subject: [PATCH 39/47] Syntax for configuring logs print color in `profiles.yml` --- website/docs/reference/global-configs/print-output.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/website/docs/reference/global-configs/print-output.md b/website/docs/reference/global-configs/print-output.md index 83280677229..dfd256628f2 100644 --- a/website/docs/reference/global-configs/print-output.md +++ b/website/docs/reference/global-configs/print-output.md @@ -80,7 +80,14 @@ dbt --no-use-colors run ``` -You can set the color preferences for the file logs only using the `--use-colors-file / --no-use-colors-file` flags. +You can set the color preferences for the file logs only within `profiles.yml` or using the `--use-colors-file / --no-use-colors-file` flags. + + + +```yaml +config: + use_colors_file: False +``` ```text dbt --use-colors-file run @@ -89,4 +96,4 @@ dbt --no-use-colors-file run - \ No newline at end of file + From 33ccc9449aea9748670979cfa18c8833961cf459 Mon Sep 17 00:00:00 2001 From: Doug Beatty <44704949+dbeatty10@users.noreply.github.com> Date: Fri, 29 Sep 2023 18:09:05 -0600 Subject: [PATCH 40/47] Syntax for configuring logs print color in `profiles.yml` --- website/docs/reference/global-configs/logs.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/website/docs/reference/global-configs/logs.md b/website/docs/reference/global-configs/logs.md index f5f1b3f814b..239f36bed86 100644 --- a/website/docs/reference/global-configs/logs.md +++ b/website/docs/reference/global-configs/logs.md @@ -124,7 +124,14 @@ dbt --quiet run ### Color -You can set the color preferences for the file logs only using the `--use-colors-file / --no-use-colors-file` flags. +You can set the color preferences for the file logs only within `profiles.yml` or using the `--use-colors-file / --no-use-colors-file` flags. + + + +```yaml +config: + use_colors_file: False +``` ```text dbt --use-colors-file run From 6f7ddda7801d281dabce2b5f45f630a3277e5287 Mon Sep 17 00:00:00 2001 From: Doug Beatty <44704949+dbeatty10@users.noreply.github.com> Date: Fri, 29 Sep 2023 19:47:13 -0600 Subject: [PATCH 41/47] Fix closing tags --- website/docs/reference/global-configs/print-output.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/website/docs/reference/global-configs/print-output.md b/website/docs/reference/global-configs/print-output.md index dfd256628f2..112b92b546f 100644 --- a/website/docs/reference/global-configs/print-output.md +++ b/website/docs/reference/global-configs/print-output.md @@ -74,6 +74,8 @@ config: use_colors: False ``` + + ```text dbt --use-colors run dbt --no-use-colors run @@ -89,11 +91,11 @@ config: use_colors_file: False ``` + + ```text dbt --use-colors-file run dbt --no-use-colors-file run ``` - - From 5fdb8b601598a93bcedf471c99a589eb1d04ea33 Mon Sep 17 00:00:00 2001 From: Doug Beatty <44704949+dbeatty10@users.noreply.github.com> Date: Fri, 29 Sep 2023 19:50:34 -0600 Subject: [PATCH 42/47] Fix closing tags --- website/docs/reference/global-configs/logs.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/website/docs/reference/global-configs/logs.md b/website/docs/reference/global-configs/logs.md index 239f36bed86..a74d79e074a 100644 --- a/website/docs/reference/global-configs/logs.md +++ b/website/docs/reference/global-configs/logs.md @@ -14,6 +14,9 @@ The `LOG_FORMAT` config specifies how dbt's logs should be formatted. If the val dbt --log-format json run {"code": "A001", "data": {"v": "=1.0.0"}, "invocation_id": "1193e449-4b7a-4eb1-8e8e-047a8b3b7973", "level": "info", "log_version": 1, "msg": "Running with dbt=1.0.0", "node_info": {}, "pid": 35098, "thread_name": "MainThread", "ts": "2021-12-03T10:46:59.928217Z", "type": "log_line"} ``` + + + To set the `LOG_FORMAT_FILE` type output for the file without impacting the console log format, use the `log-format-file` flag. @@ -133,6 +136,8 @@ config: use_colors_file: False ``` + + ```text dbt --use-colors-file run dbt --no-use-colors-file run From fcdb7ef7903e037b83411a23d84508f63cf50a93 Mon Sep 17 00:00:00 2001 From: Doug Beatty <44704949+dbeatty10@users.noreply.github.com> Date: Sat, 30 Sep 2023 10:25:21 -0600 Subject: [PATCH 43/47] Fix closing tags --- website/docs/reference/global-configs/logs.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/website/docs/reference/global-configs/logs.md b/website/docs/reference/global-configs/logs.md index a74d79e074a..8c819193fc6 100644 --- a/website/docs/reference/global-configs/logs.md +++ b/website/docs/reference/global-configs/logs.md @@ -40,8 +40,6 @@ See [structured logging](/reference/events-logging#structured-logging) for more ::: - - ### Log Level From b401b6c3b8e820eb14bd0c5ba9273cedc35b852d Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Mon, 2 Oct 2023 10:43:49 +0100 Subject: [PATCH 44/47] Update website/docs/docs/dbt-cloud-apis/sl-graphql.md --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index d62e16f1334..eebc7b0855b 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -27,7 +27,7 @@ The dbt Semantic Layer GraphQL API allows you to explore and query metrics and d dbt Partners can use the Semantic Layer GraphQL API to build and integration with the dbt Semantic Layer. ## Requirements to use the GraphQL API -- A dbt cloud project on dbt v1.6 or higher +- A dbt Cloud project on dbt v1.6 or higher - Metrics are defined and configured - A dbt Cloud [service token](/docs/dbt-cloud-apis/service-tokens) that has Semantic Layer access - Your dbt project is configured and connected to a data platform From 3361805329f30f232810e75043d1564aa942b2f0 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Mon, 2 Oct 2023 10:47:40 +0100 Subject: [PATCH 45/47] Update website/docs/docs/dbt-cloud-apis/sl-graphql.md --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index eebc7b0855b..f8867393197 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -29,7 +29,7 @@ dbt Partners can use the Semantic Layer GraphQL API to build and integration wit ## Requirements to use the GraphQL API - A dbt Cloud project on dbt v1.6 or higher - Metrics are defined and configured -- A dbt Cloud [service token](/docs/dbt-cloud-apis/service-tokens) that has Semantic Layer access +- A dbt Cloud [service token](/docs/dbt-cloud-apis/service-tokens with "Semantic Layer Only” and "Metadata Only" permissions - Your dbt project is configured and connected to a data platform From 184e874cbba50ba367f6a6bdf48eecdf1e5921dd Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Mon, 2 Oct 2023 10:48:18 +0100 Subject: [PATCH 46/47] Update website/docs/guides/dbt-ecosystem/sl-partner-integration-guide.md --- .../docs/guides/dbt-ecosystem/sl-partner-integration-guide.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/dbt-ecosystem/sl-partner-integration-guide.md b/website/docs/guides/dbt-ecosystem/sl-partner-integration-guide.md index b0718a6484d..68037bfd0cd 100644 --- a/website/docs/guides/dbt-ecosystem/sl-partner-integration-guide.md +++ b/website/docs/guides/dbt-ecosystem/sl-partner-integration-guide.md @@ -114,7 +114,7 @@ For better analysis, it's best to have the context of the metrics close to where These are recommendations on how to evolve a Semantic Layer integration and not a strict runbook. **Stage 1 - The basic** -* Supporting and using the new [JDBC](/docs/dbt-cloud-apis/sl-jdbc) or [GraphQL](/docs/dbt-cloud-apis/sl-graphql) is the first step. Refer to the [dbt Semantic Layer APIs](/docs/dbt-cloud-apis/sl-api-overview) for more technical details. +* Supporting and using [JDBC](/docs/dbt-cloud-apis/sl-jdbc) or [GraphQL](/docs/dbt-cloud-apis/sl-graphql) is the first step. Refer to the [dbt Semantic Layer APIs](/docs/dbt-cloud-apis/sl-api-overview) for more technical details. **Stage 2 - More discoverability and basic querying** * Support listing metrics defined in the project From 2f2554c93f73b34a6e5f1764eb3adc7af82b02c9 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Mon, 2 Oct 2023 14:16:09 +0100 Subject: [PATCH 47/47] Update sl-graphql.md fix link --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index f8867393197..3e06df69f76 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -18,18 +18,18 @@ import LegacyInfo from '/snippets/_legacy-sl-callout.md'; With GraphQL, users can request specific data using a single query, reducing the need for many server round trips. This improves performance and minimizes network overhead. -GraphQL has several advantages, such as self-documenting, having a strong typing system, supporting versioning and evolution, enabling rapid development, and having a robust ecosystem. These features make GraphQL a powerful choice for APIs that prioritize flexibility, performance, and developer productivity. +GraphQL has several advantages, such as self-documenting, having a strong typing system, supporting versioning and evolution, enabling rapid development, and having a robust ecosystem. These features make GraphQL a powerful choice for APIs prioritizing flexibility, performance, and developer productivity. ## dbt Semantic Layer GraphQL API -The dbt Semantic Layer GraphQL API allows you to explore and query metrics and dimensions. Due to it's self-documenting nature, you can explore the calls conveniently through the [schema explorer](https://semantic-layer.cloud.getdbt.com/api/graphql). +The dbt Semantic Layer GraphQL API allows you to explore and query metrics and dimensions. Due to its self-documenting nature, you can explore the calls conveniently through the [schema explorer](https://semantic-layer.cloud.getdbt.com/api/graphql). -dbt Partners can use the Semantic Layer GraphQL API to build and integration with the dbt Semantic Layer. +dbt Partners can use the Semantic Layer GraphQL API to build an integration with the dbt Semantic Layer. ## Requirements to use the GraphQL API - A dbt Cloud project on dbt v1.6 or higher - Metrics are defined and configured -- A dbt Cloud [service token](/docs/dbt-cloud-apis/service-tokens with "Semantic Layer Only” and "Metadata Only" permissions +- A dbt Cloud [service token](/docs/dbt-cloud-apis/service-tokens) with "Semantic Layer Only” and "Metadata Only" permissions - Your dbt project is configured and connected to a data platform @@ -289,7 +289,7 @@ mutation { } ``` -Note that when using granularity in the query, the output of a time dimension with a time grain applied to it always takes form of dimension name appended with a double underscore and the granularity level - `{time_dimension_name}__{DAY|WEEK|MONTH|QUARTER|YEAR}`. Even if no granularity is specified, it will also always have a granularity appended to it and will default to the lowest available (usually daily for most data sources). It is encouraged to specify a granularity when using time dimensions so that there won't be any unexpected results with the output data. +Note that when using granularity in the query, the output of a time dimension with a time grain applied to it always takes the form of a dimension name appended with a double underscore and the granularity level - `{time_dimension_name}__{DAY|WEEK|MONTH|QUARTER|YEAR}`. Even if no granularity is specified, it will also always have a granularity appended to it and will default to the lowest available (usually daily for most data sources). It is encouraged to specify a granularity when using time dimensions so that there won't be any unexpected results with the output data. **Query two metrics with a categorical dimension** @@ -307,7 +307,7 @@ mutation { **Query with a where filter** -The `where` filter takes a list argument (or a string for a single input). Depending on the object you are filtering on, there are a couple of parameters: +The `where` filter takes a list argument (or a string for a single input). Depending on the object you are filtering, there are a couple of parameters: - `Dimension()` — Used for any categorical or time dimensions. If used for a time dimension, granularity is required. For example, `Dimension('metric_time').grain('week')` or `Dimension('customer__country')`. @@ -459,4 +459,4 @@ order_total ordered_at 12 2023-08-09 5123 2023-08-10 """ -``` \ No newline at end of file +```