From 36843e83af6b0882b3f7596b2bbe54afaa80441c Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Tue, 2 Jan 2024 15:24:55 -0500 Subject: [PATCH 1/7] add note about ref --- .../docs/dbt-cloud-apis/sl-api-overview.md | 2 +- .../docs/docs/dbt-cloud-apis/sl-graphql.md | 2 ++ website/docs/docs/dbt-cloud-apis/sl-jdbc.md | 2 ++ website/docs/guides/sl-migration.md | 19 ++++++++++++------- 4 files changed, 17 insertions(+), 8 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-api-overview.md b/website/docs/docs/dbt-cloud-apis/sl-api-overview.md index 6644d3e4b8b..0ddbc6888db 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-api-overview.md +++ b/website/docs/docs/dbt-cloud-apis/sl-api-overview.md @@ -15,7 +15,7 @@ import DeprecationNotice from '/snippets/_sl-deprecation-notice.md'; -The rapid growth of different tools in the modern data stack has helped data professionals address the diverse needs of different teams. The downside of this growth is the fragmentation of business logic across teams, tools, and workloads. +The rapid growth of different tools in the modern data stack has helped data professionals address the diverse needs of different teams. The downside of this growth is the fragmentation of business logic across teams, tools, and workloads.
The [dbt Semantic Layer](/docs/use-dbt-semantic-layer/dbt-sl) allows you to define metrics in code (with [MetricFlow](/docs/build/about-metricflow)) and dynamically generate and query datasets in downstream tools based on their dbt governed assets, such as metrics and models. Integrating with the dbt Semantic Layer will help organizations that use your product make more efficient and trustworthy decisions with their data. It also helps you to avoid duplicative coding, optimize development workflow, ensure data governance, and guarantee consistency for data consumers. diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 3555b211f4f..0898d75762a 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -26,6 +26,8 @@ The dbt Semantic Layer GraphQL API allows you to explore and query metrics and d dbt Partners can use the Semantic Layer GraphQL API to build an integration with the dbt Semantic Layer. +Note, the dbt Semantic Layer API doesn't support `ref` to call dbt objects. Instead, use the complete qualified table name. If you're using dbt macros at query time to calculate your metrics, you should move those calculations into the Semantic Layer as code. + ## Requirements to use the GraphQL API - A dbt Cloud project on dbt v1.6 or higher - Metrics are defined and configured diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md index 345be39635e..61ba6bc63c8 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md +++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md @@ -33,6 +33,8 @@ You *may* be able to use our JDBC API with tools that do not have an official in Refer to [Get started with the dbt Semantic Layer](/docs/use-dbt-semantic-layer/quickstart-sl) for more info. +Note, the dbt Semantic Layer API doesn't support `ref` to call dbt objects. Instead, use the complete qualified table name. If you're using dbt macros at query time to calculate your metrics, you should move those calculations into the Semantic Layer as code. + ## Authentication dbt Cloud authorizes requests to the dbt Semantic Layer API. You need to provide an environment ID, host, and [service account tokens](/docs/dbt-cloud-apis/service-tokens). diff --git a/website/docs/guides/sl-migration.md b/website/docs/guides/sl-migration.md index 8ede40a6a2d..184f66ec731 100644 --- a/website/docs/guides/sl-migration.md +++ b/website/docs/guides/sl-migration.md @@ -25,21 +25,26 @@ dbt Labs recommends completing these steps in a local dev environment (such as t 1. Create new Semantic Model configs as YAML files in your dbt project.* 1. Upgrade the metrics configs in your project to the new spec.* 1. Delete your old metrics file or remove the `.yml` file extension so they're ignored at parse time. Remove the `dbt-metrics` package from your project. Remove any macros that reference `dbt-metrics`, like `metrics.calculate()`. Make sure that any packages you’re using don't have references to the old metrics spec. -1. Install the CLI with `python -m pip install "dbt-metricflow[your_adapter_name]"`. For example: +1. Install the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) to run MetricFlow commands and define your semantic model configurations. + - If you're using dbt Core, you can install the [MetricFlow CLI](/docs/build/metricflow-commands) with `python -m pip install "dbt-metricflow[your_adapter_name]"`. For example: ```bash python -m pip install "dbt-metricflow[snowflake]" ``` - **Note** - The MetricFlow CLI is not available in the IDE at this time. Support is coming soon. + **Note** - MetricFlow commands aren't yet supported in the dbt CLoud IDE at this time. -1. Run `dbt parse`. This parses your project and creates a `semantic_manifest.json` file in your target directory. MetricFlow needs this file to query metrics. If you make changes to your configs, you will need to parse your project again. -1. Run `mf list metrics` to view the metrics in your project. -1. Test querying a metric by running `mf query --metrics --group-by `. For example: +2. Run `dbt parse`. This parses your project and creates a `semantic_manifest.json` file in your target directory. MetricFlow needs this file to query metrics. If you make changes to your configs, you will need to parse your project again. +3. Run `mf list metrics` to view the metrics in your project. +4. Test querying a metric by running `mf query --metrics --group-by `. For example: ```bash mf query --metrics revenue --group-by metric_time ``` -1. Run `mf validate-configs` to run semantic and warehouse validations. This ensures your configs are valid and the underlying objects exist in your warehouse. -1. Push these changes to a new branch in your repo. +5. Run `mf validate-configs` to run semantic and warehouse validations. This ensures your configs are valid and the underlying objects exist in your warehouse. +6. Push these changes to a new branch in your repo. + +:::info `ref` not supported +The dbt Semantic Layer API doesn't support `ref` to call dbt objects. Instead, use the complete qualified table name. If you're using dbt macros at query time to calculate your metrics, you should move those calculations into the Semantic Layer as code. +::: **To make this process easier, dbt Labs provides a [custom migration tool](https://github.com/dbt-labs/dbt-converter) that automates these steps for you. You can find installation instructions in the [README](https://github.com/dbt-labs/dbt-converter/blob/master/README.md). Derived metrics aren’t supported in the migration tool, and will have to be migrated manually.* From 42ea5c9d1678ba8a289fc3bf2fda915ec596f575 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Tue, 2 Jan 2024 15:28:38 -0500 Subject: [PATCH 2/7] add why --- website/docs/guides/sl-migration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/sl-migration.md b/website/docs/guides/sl-migration.md index 184f66ec731..69c964882f1 100644 --- a/website/docs/guides/sl-migration.md +++ b/website/docs/guides/sl-migration.md @@ -43,7 +43,7 @@ dbt Labs recommends completing these steps in a local dev environment (such as t 6. Push these changes to a new branch in your repo. :::info `ref` not supported -The dbt Semantic Layer API doesn't support `ref` to call dbt objects. Instead, use the complete qualified table name. If you're using dbt macros at query time to calculate your metrics, you should move those calculations into the Semantic Layer as code. +The dbt Semantic Layer API doesn't support `ref` to call dbt objects. This is currently due to differences in architecture between the legacy Semantic Layer and the re-released Semantic Layer. Instead, use the complete qualified table name. If you're using dbt macros at query time to calculate your metrics, you should move those calculations into the Semantic Layer as code. ::: **To make this process easier, dbt Labs provides a [custom migration tool](https://github.com/dbt-labs/dbt-converter) that automates these steps for you. You can find installation instructions in the [README](https://github.com/dbt-labs/dbt-converter/blob/master/README.md). Derived metrics aren’t supported in the migration tool, and will have to be migrated manually.* From 255fadc40f06c136ba48a674da6759f550cb3733 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Tue, 2 Jan 2024 15:29:33 -0500 Subject: [PATCH 3/7] add currently --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 2 +- website/docs/docs/dbt-cloud-apis/sl-jdbc.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 0898d75762a..6a92e11d7c0 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -26,7 +26,7 @@ The dbt Semantic Layer GraphQL API allows you to explore and query metrics and d dbt Partners can use the Semantic Layer GraphQL API to build an integration with the dbt Semantic Layer. -Note, the dbt Semantic Layer API doesn't support `ref` to call dbt objects. Instead, use the complete qualified table name. If you're using dbt macros at query time to calculate your metrics, you should move those calculations into the Semantic Layer as code. +Note, the dbt Semantic Layer API doesn't currently support `ref` to call dbt objects. Instead, use the complete qualified table name. If you're using dbt macros at query time to calculate your metrics, you should move those calculations into the Semantic Layer as code. ## Requirements to use the GraphQL API - A dbt Cloud project on dbt v1.6 or higher diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md index 61ba6bc63c8..6ec916b7719 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md +++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md @@ -33,7 +33,7 @@ You *may* be able to use our JDBC API with tools that do not have an official in Refer to [Get started with the dbt Semantic Layer](/docs/use-dbt-semantic-layer/quickstart-sl) for more info. -Note, the dbt Semantic Layer API doesn't support `ref` to call dbt objects. Instead, use the complete qualified table name. If you're using dbt macros at query time to calculate your metrics, you should move those calculations into the Semantic Layer as code. +Note, the dbt Semantic Layer API doesn't currently support `ref` to call dbt objects. Instead, use the complete qualified table name. If you're using dbt macros at query time to calculate your metrics, you should move those calculations into the Semantic Layer as code. ## Authentication From 7d00d4db3242b4c035071e39b52f8c1d47bf2390 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 3 Jan 2024 07:45:55 -0500 Subject: [PATCH 4/7] Update website/docs/docs/dbt-cloud-apis/sl-graphql.md --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 6a92e11d7c0..56a4ac7ba59 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -26,7 +26,7 @@ The dbt Semantic Layer GraphQL API allows you to explore and query metrics and d dbt Partners can use the Semantic Layer GraphQL API to build an integration with the dbt Semantic Layer. -Note, the dbt Semantic Layer API doesn't currently support `ref` to call dbt objects. Instead, use the complete qualified table name. If you're using dbt macros at query time to calculate your metrics, you should move those calculations into the Semantic Layer as code. +Note that the dbt Semantic Layer API doesn't support `ref` to call dbt objects. Instead, use the complete qualified table name. If you're using dbt macros at query time to calculate your metrics, you should move those calculations into your Semantic Layer metric definitions as code. ## Requirements to use the GraphQL API - A dbt Cloud project on dbt v1.6 or higher From 0af829d93d0fae1f27eb80c616f01cddecf68284 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 3 Jan 2024 07:46:09 -0500 Subject: [PATCH 5/7] Update website/docs/docs/dbt-cloud-apis/sl-jdbc.md --- website/docs/docs/dbt-cloud-apis/sl-jdbc.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md index 6ec916b7719..97f70902c74 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md +++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md @@ -33,7 +33,7 @@ You *may* be able to use our JDBC API with tools that do not have an official in Refer to [Get started with the dbt Semantic Layer](/docs/use-dbt-semantic-layer/quickstart-sl) for more info. -Note, the dbt Semantic Layer API doesn't currently support `ref` to call dbt objects. Instead, use the complete qualified table name. If you're using dbt macros at query time to calculate your metrics, you should move those calculations into the Semantic Layer as code. +Note that the dbt Semantic Layer API doesn't support `ref` to call dbt objects. Instead, use the complete qualified table name. If you're using dbt macros at query time to calculate your metrics, you should move those calculations into your Semantic Layer metric definitions as code. ## Authentication From 53d7c83141a921f59f506d4f3cca5819417c321a Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 3 Jan 2024 07:46:45 -0500 Subject: [PATCH 6/7] Update website/docs/guides/sl-migration.md --- website/docs/guides/sl-migration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/sl-migration.md b/website/docs/guides/sl-migration.md index 69c964882f1..1e68946c8b3 100644 --- a/website/docs/guides/sl-migration.md +++ b/website/docs/guides/sl-migration.md @@ -43,7 +43,7 @@ dbt Labs recommends completing these steps in a local dev environment (such as t 6. Push these changes to a new branch in your repo. :::info `ref` not supported -The dbt Semantic Layer API doesn't support `ref` to call dbt objects. This is currently due to differences in architecture between the legacy Semantic Layer and the re-released Semantic Layer. Instead, use the complete qualified table name. If you're using dbt macros at query time to calculate your metrics, you should move those calculations into the Semantic Layer as code. +The dbt Semantic Layer API doesn't support `ref` to call dbt objects. This is currently due to differences in architecture between the legacy Semantic Layer and the re-released Semantic Layer. Instead, use the complete qualified table name. If you're using dbt macros at query time to calculate your metrics, you should move those calculations into your Semantic Layer metric definitions as code. ::: **To make this process easier, dbt Labs provides a [custom migration tool](https://github.com/dbt-labs/dbt-converter) that automates these steps for you. You can find installation instructions in the [README](https://github.com/dbt-labs/dbt-converter/blob/master/README.md). Derived metrics aren’t supported in the migration tool, and will have to be migrated manually.* From f540fcbbcd457f89e87b4d79594cb3315ba74852 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 3 Jan 2024 09:39:24 -0500 Subject: [PATCH 7/7] Update website/docs/guides/sl-migration.md Co-authored-by: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> --- website/docs/guides/sl-migration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/sl-migration.md b/website/docs/guides/sl-migration.md index 1e68946c8b3..afa181646e3 100644 --- a/website/docs/guides/sl-migration.md +++ b/website/docs/guides/sl-migration.md @@ -26,7 +26,7 @@ dbt Labs recommends completing these steps in a local dev environment (such as t 1. Upgrade the metrics configs in your project to the new spec.* 1. Delete your old metrics file or remove the `.yml` file extension so they're ignored at parse time. Remove the `dbt-metrics` package from your project. Remove any macros that reference `dbt-metrics`, like `metrics.calculate()`. Make sure that any packages you’re using don't have references to the old metrics spec. 1. Install the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) to run MetricFlow commands and define your semantic model configurations. - - If you're using dbt Core, you can install the [MetricFlow CLI](/docs/build/metricflow-commands) with `python -m pip install "dbt-metricflow[your_adapter_name]"`. For example: + - If you're using dbt Core, install the [MetricFlow CLI](/docs/build/metricflow-commands) with `python -m pip install "dbt-metricflow[your_adapter_name]"`. For example: ```bash python -m pip install "dbt-metricflow[snowflake]"