From 7b07a591bcc61084fd9b2c9310b4deff14de3f35 Mon Sep 17 00:00:00 2001 From: Alex Coleman Date: Wed, 11 Dec 2024 15:42:55 +0000 Subject: [PATCH 01/12] Correct bind mount for profiles.yml Corrects the bind mount for profiles.yml which previously mounted the host profiles.yml file to /root/.dbt. --- website/docs/docs/core/docker-install.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/core/docker-install.md b/website/docs/docs/core/docker-install.md index 5ce033c7996..3cf066e2231 100644 --- a/website/docs/docs/core/docker-install.md +++ b/website/docs/docs/core/docker-install.md @@ -34,7 +34,7 @@ The `ENTRYPOINT` for dbt Docker images is the command `dbt`. You can bind-mount docker run \ --network=host \ --mount type=bind,source=path/to/project,target=/usr/app \ ---mount type=bind,source=path/to/profiles.yml,target=/root/.dbt/ \ +--mount type=bind,source=path/to/profiles.yml,target=/root/.dbt/profiles.yml \ \ ls ``` From 0c9a5f224a850d584b0d67adeeca58a7a94b9b27 Mon Sep 17 00:00:00 2001 From: rcg798 <5733407+rcg798@users.noreply.github.com> Date: Mon, 16 Dec 2024 10:05:42 -0800 Subject: [PATCH 02/12] Update job-scheduler.md Coming newly into the orchestration team, I am updating the initial description of the scheduler types to better align with how users may view the product today: 1. we refer to merge jobs as merge jobs (and it is an explicit category in the UI) so I think it would be useful to call out that term in the beginning, so people associate that type of job with its name and 2. another event-based job is job completion (another explicit category in the UI) and I couldn't tell how that fit into the initial description, so added that as well. Feel free to revise, just wanted to flag those two points! --- website/docs/docs/deploy/job-scheduler.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/website/docs/docs/deploy/job-scheduler.md b/website/docs/docs/deploy/job-scheduler.md index 7d45fddc3f6..fb351703624 100644 --- a/website/docs/docs/deploy/job-scheduler.md +++ b/website/docs/docs/deploy/job-scheduler.md @@ -11,7 +11,8 @@ The job scheduler is the backbone of running jobs in dbt Cloud, bringing power a The scheduler enables both cron-based and event-driven execution of dbt commands in the user’s data platform. Specifically, it handles: - Cron-based execution of dbt Cloud jobs that run on a predetermined cadence -- Event-driven execution of dbt Cloud CI jobs triggered by pull requests to the dbt repo +- Event-driven execution of dbt Cloud jobs that run based on the completion of another job +- Event-driven execution of dbt Cloud CI jobs triggered by pull requests to the dbt repo (e.g. "merge jobs") - Event-driven execution of dbt Cloud jobs triggered by API - Event-driven execution of dbt Cloud jobs manually triggered by a user to "Run Now" From a7e7e1bc9f6d9a850f4c13c790106c516d7ffa53 Mon Sep 17 00:00:00 2001 From: bethanyhipple-dbtlabs <108838013+bethanyhipple-dbtlabs@users.noreply.github.com> Date: Mon, 16 Dec 2024 14:58:14 -0800 Subject: [PATCH 03/12] Remove extra fields in create table statements The CSV files for orders and payment don't have _etl_loaded_at and batched_at respectively. So removing these fields from the create table statements. --- website/docs/guides/redshift-qs.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/website/docs/guides/redshift-qs.md b/website/docs/guides/redshift-qs.md index 83fafad1d12..8990c4db925 100644 --- a/website/docs/guides/redshift-qs.md +++ b/website/docs/guides/redshift-qs.md @@ -118,8 +118,7 @@ Now we are going to load our sample data into the S3 bucket that our Cloudformat id integer, user_id integer, order_date date, - status varchar(50), - _etl_loaded_at timestamp default current_timestamp + status varchar(50) ); create table stripe.payment( @@ -128,8 +127,7 @@ Now we are going to load our sample data into the S3 bucket that our Cloudformat paymentmethod varchar(50), status varchar(50), amount integer, - created date, - _batched_at timestamp default current_timestamp + created date ); ``` From da5891bdd869e78a9a92ca8e5cbf1b01fd6db248 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 17 Dec 2024 00:45:29 +0000 Subject: [PATCH 04/12] add azure st support (#6667) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Merge: Dec 15th or 16th this pr clarifies that azure ST is supported for SL - added to the tenancy table - added to the prerequs [ ] needs PM review [ x ] needs docs review [docs notion}(https://www.notion.so/dbtlabs/SL-support-in-Azure-ST-15abb38ebda780ce8ed1dd34843bf413?pvs=4) --- πŸš€ Deployment available! Here are the direct links to the updated files: - https://docs-getdbt-com-git-sl-azure-st-dbt-labs.vercel.app/docs/dbt-versions/release-notes --------- Co-authored-by: Leona B. Campbell <3880403+runleonarun@users.noreply.github.com> --- website/docs/docs/dbt-versions/release-notes.md | 1 + website/snippets/_v2-sl-prerequisites.md | 5 +++-- website/snippets/cloud-feature-parity.md | 2 +- 3 files changed, 5 insertions(+), 3 deletions(-) diff --git a/website/docs/docs/dbt-versions/release-notes.md b/website/docs/docs/dbt-versions/release-notes.md index b83542ef10e..fdc92a32ff9 100644 --- a/website/docs/docs/dbt-versions/release-notes.md +++ b/website/docs/docs/dbt-versions/release-notes.md @@ -20,6 +20,7 @@ Release notes are grouped by month for both multi-tenant and virtual private clo ## December 2024 +- **New**: The dbt Semantic Layer now supports Azure Single-tenant deployments. Refer to [Set up the dbt Semantic Layer](/docs/use-dbt-semantic-layer/setup-sl) for more information on how to get started. - **Fix**: Resolved intermittent issues in Single-tenant environments affecting Semantic Layer and query history. - **Fix**: [The dbt Semantic Layer](/docs/use-dbt-semantic-layer/dbt-sl) now respects the BigQuery [`execution_project` attribute](/docs/core/connect-data-platform/bigquery-setup#execution-project), including for exports. - **New**: [Model notifications](/docs/deploy/model-notifications) are now generally available in dbt Cloud. These notifications alert model owners through email about any issues encountered by models and tests as soon as they occur while running a job. diff --git a/website/snippets/_v2-sl-prerequisites.md b/website/snippets/_v2-sl-prerequisites.md index 68f6e7d10b7..4ad7ff6b755 100644 --- a/website/snippets/_v2-sl-prerequisites.md +++ b/website/snippets/_v2-sl-prerequisites.md @@ -1,5 +1,6 @@ -- Have a dbt Cloud Team or Enterprise account. Single-tenant accounts should contact their account representative for setup. -- Ensure your production and development environments use [dbt version 1.6 or higher](/docs/dbt-versions/upgrade-dbt-version-in-cloud). +- Have a dbt Cloud Team or Enterprise account. + - Available on all [tenant configurations](/docs/cloud/about-cloud/tenancy). Single-tenant accounts should contact your account representative for setup. +- Ensure your production and development environments are on a [supported dbt version](/docs/dbt-versions/upgrade-dbt-version-in-cloud). - Use Snowflake, BigQuery, Databricks, or Redshift. - Create a successful run in the environment where you configure the Semantic Layer. - **Note:** Semantic Layer supports querying in Deployment environments; development querying is coming soon. diff --git a/website/snippets/cloud-feature-parity.md b/website/snippets/cloud-feature-parity.md index 9348f4d409b..1107b999c14 100644 --- a/website/snippets/cloud-feature-parity.md +++ b/website/snippets/cloud-feature-parity.md @@ -9,7 +9,7 @@ The following table outlines which dbt Cloud features are supported on the diffe | dbt Copilot | βœ… | βœ… | βœ… | βœ… | | dbt Explorer | βœ… | βœ… | βœ… | βœ… | | dbt Mesh | βœ… | βœ… | βœ… | βœ… | -| dbt Semantic Layer | βœ… | βœ… (Upon request) | βœ… | ❌ | +| dbt Semantic Layer | βœ… | βœ… (Upon request) | βœ… | βœ… | | Discovery API | βœ… | βœ… | βœ… | βœ… | | IP restrictions | βœ… | βœ… | βœ… | βœ… | | Job scheduler | βœ… | βœ… | βœ… | βœ… | From 5980a47e2ae759aec262f46fb3c32e34130ab911 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 17 Dec 2024 00:50:47 +0000 Subject: [PATCH 05/12] adds sigma as SL integration (#6670) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit this PR adds sigma as an SL integration (available in preview) [ ] needs pm review [ ] needs docs review (optional) --- πŸš€ Deployment available! Here are the direct links to the updated files: - https://docs-getdbt-com-git-sl-sigma-preview-dbt-labs.vercel.app/docs/dbt-versions/release-notes --------- Co-authored-by: Leona B. Campbell <3880403+runleonarun@users.noreply.github.com> --- .../docs/docs/dbt-versions/release-notes.md | 1 + website/snippets/_sl-partner-links.md | 15 ++ website/static/img/icons/question-mark.svg | 2 +- website/static/img/icons/sigma.svg | 185 ++++++++++++++++++ website/static/img/icons/white/redshift.svg | 1 - website/static/img/icons/white/rocket.svg | 1 - website/static/img/icons/white/sigma.svg | 185 ++++++++++++++++++ 7 files changed, 387 insertions(+), 3 deletions(-) create mode 100644 website/static/img/icons/sigma.svg delete mode 100644 website/static/img/icons/white/redshift.svg delete mode 100644 website/static/img/icons/white/rocket.svg create mode 100644 website/static/img/icons/white/sigma.svg diff --git a/website/docs/docs/dbt-versions/release-notes.md b/website/docs/docs/dbt-versions/release-notes.md index fdc92a32ff9..c1e8de6f1ad 100644 --- a/website/docs/docs/dbt-versions/release-notes.md +++ b/website/docs/docs/dbt-versions/release-notes.md @@ -20,6 +20,7 @@ Release notes are grouped by month for both multi-tenant and virtual private clo ## December 2024 +- **New**: The dbt Semantic Layer supports Sigma as a [partner integration](/docs/cloud-integrations/avail-sl-integrations), available in Preview. Refer to [Sigma](https://help.sigmacomputing.com/docs/configure-a-dbt-semantic-layer-integration) for more information. - **New**: The dbt Semantic Layer now supports Azure Single-tenant deployments. Refer to [Set up the dbt Semantic Layer](/docs/use-dbt-semantic-layer/setup-sl) for more information on how to get started. - **Fix**: Resolved intermittent issues in Single-tenant environments affecting Semantic Layer and query history. - **Fix**: [The dbt Semantic Layer](/docs/use-dbt-semantic-layer/dbt-sl) now respects the BigQuery [`execution_project` attribute](/docs/core/connect-data-platform/bigquery-setup#execution-project), including for exports. diff --git a/website/snippets/_sl-partner-links.md b/website/snippets/_sl-partner-links.md index 7d08323239b..7c5f376180b 100644 --- a/website/snippets/_sl-partner-links.md +++ b/website/snippets/_sl-partner-links.md @@ -106,6 +106,21 @@ The following tools integrate with the dbt Semantic Layer: +
+ + + + +
+ +
\ No newline at end of file + diff --git a/website/static/img/icons/sigma.svg b/website/static/img/icons/sigma.svg new file mode 100644 index 00000000000..411c2a74b46 --- /dev/null +++ b/website/static/img/icons/sigma.svg @@ -0,0 +1,185 @@ + + + + + + + + + diff --git a/website/static/img/icons/white/redshift.svg b/website/static/img/icons/white/redshift.svg deleted file mode 100644 index e57888813e3..00000000000 --- a/website/static/img/icons/white/redshift.svg +++ /dev/null @@ -1 +0,0 @@ - \ No newline at end of file diff --git a/website/static/img/icons/white/rocket.svg b/website/static/img/icons/white/rocket.svg deleted file mode 100644 index 50f06851db9..00000000000 --- a/website/static/img/icons/white/rocket.svg +++ /dev/null @@ -1 +0,0 @@ - \ No newline at end of file diff --git a/website/static/img/icons/white/sigma.svg b/website/static/img/icons/white/sigma.svg new file mode 100644 index 00000000000..411c2a74b46 --- /dev/null +++ b/website/static/img/icons/white/sigma.svg @@ -0,0 +1,185 @@ + + + + + + + + + From ae88a035f1a03d58eb54e63e07bba79e2903d96e Mon Sep 17 00:00:00 2001 From: erdinculuturk-nw <101416454+erdinculuturk-nw@users.noreply.github.com> Date: Tue, 17 Dec 2024 10:06:02 +0100 Subject: [PATCH 06/12] Update model-notifications.md - description is not allowed as a group property 08:32:37 Running dbt... dbt command failed 08:32:58 Encountered an error: Parsing Error Invalid groups config given in ...... Additional properties are not allowed ('description' was unexpected) --- website/docs/docs/deploy/model-notifications.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/website/docs/docs/deploy/model-notifications.md b/website/docs/docs/deploy/model-notifications.md index 24bbc2295c6..45ffbef0a4f 100644 --- a/website/docs/docs/deploy/model-notifications.md +++ b/website/docs/docs/deploy/model-notifications.md @@ -36,17 +36,17 @@ version: 2 groups: - name: finance - description: "Models related to the finance department" owner: # Email is required to receive model-level notifications, additional properties are also allowed. name: "Finance Team" + description: "Models related to the finance department" email: finance@dbtlabs.com favorite_food: donuts - name: marketing - description: "Models related to the marketing department" owner: name: "Marketing Team" + description: "Models related to the marketing department" email: marketing@dbtlabs.com favorite_food: jaffles ``` From 0fd24b183b9640f88739e337dfd0c31a7b10dfc2 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 17 Dec 2024 10:20:15 +0000 Subject: [PATCH 07/12] Update docker-install.md both options work as confirmed by @jeremyyeo --- website/docs/docs/core/docker-install.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/website/docs/docs/core/docker-install.md b/website/docs/docs/core/docker-install.md index 3cf066e2231..fc3caa42a03 100644 --- a/website/docs/docs/core/docker-install.md +++ b/website/docs/docs/core/docker-install.md @@ -30,6 +30,7 @@ docker pull ghcr.io/dbt-labs/: ### Running a dbt Docker image in a container The `ENTRYPOINT` for dbt Docker images is the command `dbt`. You can bind-mount your project to `/usr/app` and use dbt as normal: + ``` docker run \ --network=host \ @@ -39,6 +40,17 @@ docker run \ ls ``` +Or + +``` +docker run \ +--network=host \ +--mount type=bind,source=path/to/project,target=/usr/app \ +--mount type=bind,source=path/to/profiles.yml.dbt,target=/root/.dbt/ \ + \ +ls +``` + Notes: * Bind-mount sources _must_ be an absolute path * You may need to make adjustments to the docker networking setting depending on the specifics of your data warehouse or database host. From 6b18573cadb772d466f1ae43c713e3fb38c2e800 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 17 Dec 2024 11:24:10 +0000 Subject: [PATCH 08/12] Update website/docs/docs/deploy/job-scheduler.md --- website/docs/docs/deploy/job-scheduler.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/deploy/job-scheduler.md b/website/docs/docs/deploy/job-scheduler.md index fb351703624..e8a28bbfb27 100644 --- a/website/docs/docs/deploy/job-scheduler.md +++ b/website/docs/docs/deploy/job-scheduler.md @@ -11,7 +11,7 @@ The job scheduler is the backbone of running jobs in dbt Cloud, bringing power a The scheduler enables both cron-based and event-driven execution of dbt commands in the user’s data platform. Specifically, it handles: - Cron-based execution of dbt Cloud jobs that run on a predetermined cadence -- Event-driven execution of dbt Cloud jobs that run based on the completion of another job +- Event-driven execution of dbt Cloud jobs that [run based on the completion of another job ([trigger on job completion](/docs/deploy/deploy-jobs#trigger-on-job-completion)) - Event-driven execution of dbt Cloud CI jobs triggered by pull requests to the dbt repo (e.g. "merge jobs") - Event-driven execution of dbt Cloud jobs triggered by API - Event-driven execution of dbt Cloud jobs manually triggered by a user to "Run Now" From d079ab41aaf7ceb0b7cb7b925bdbb70ef13000d9 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 17 Dec 2024 11:24:46 +0000 Subject: [PATCH 09/12] Update website/docs/docs/deploy/job-scheduler.md --- website/docs/docs/deploy/job-scheduler.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/deploy/job-scheduler.md b/website/docs/docs/deploy/job-scheduler.md index e8a28bbfb27..38a98d2f92e 100644 --- a/website/docs/docs/deploy/job-scheduler.md +++ b/website/docs/docs/deploy/job-scheduler.md @@ -12,7 +12,7 @@ The scheduler enables both cron-based and event-driven execution of dbt commands - Cron-based execution of dbt Cloud jobs that run on a predetermined cadence - Event-driven execution of dbt Cloud jobs that [run based on the completion of another job ([trigger on job completion](/docs/deploy/deploy-jobs#trigger-on-job-completion)) -- Event-driven execution of dbt Cloud CI jobs triggered by pull requests to the dbt repo (e.g. "merge jobs") +- Event-driven execution of dbt Cloud CI jobs triggered by pull requests to the dbt repo ([merge jobs](/docs/deploy/merge-jobs)) - Event-driven execution of dbt Cloud jobs triggered by API - Event-driven execution of dbt Cloud jobs manually triggered by a user to "Run Now" From f99d323aa52885f1c77b6c83655b4a37bc200596 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 17 Dec 2024 11:24:59 +0000 Subject: [PATCH 10/12] Update website/docs/docs/deploy/job-scheduler.md --- website/docs/docs/deploy/job-scheduler.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/deploy/job-scheduler.md b/website/docs/docs/deploy/job-scheduler.md index 38a98d2f92e..f3f5706e93f 100644 --- a/website/docs/docs/deploy/job-scheduler.md +++ b/website/docs/docs/deploy/job-scheduler.md @@ -11,7 +11,7 @@ The job scheduler is the backbone of running jobs in dbt Cloud, bringing power a The scheduler enables both cron-based and event-driven execution of dbt commands in the user’s data platform. Specifically, it handles: - Cron-based execution of dbt Cloud jobs that run on a predetermined cadence -- Event-driven execution of dbt Cloud jobs that [run based on the completion of another job ([trigger on job completion](/docs/deploy/deploy-jobs#trigger-on-job-completion)) +- Event-driven execution of dbt Cloud jobs that run based on the completion of another job ([trigger on job completion](/docs/deploy/deploy-jobs#trigger-on-job-completion)) - Event-driven execution of dbt Cloud CI jobs triggered by pull requests to the dbt repo ([merge jobs](/docs/deploy/merge-jobs)) - Event-driven execution of dbt Cloud jobs triggered by API - Event-driven execution of dbt Cloud jobs manually triggered by a user to "Run Now" From 4f4bdb985ae8bfde3be0ba28482e87df9697b80b Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 17 Dec 2024 11:25:39 +0000 Subject: [PATCH 11/12] Update website/docs/docs/deploy/job-scheduler.md --- website/docs/docs/deploy/job-scheduler.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/deploy/job-scheduler.md b/website/docs/docs/deploy/job-scheduler.md index f3f5706e93f..1990832f7df 100644 --- a/website/docs/docs/deploy/job-scheduler.md +++ b/website/docs/docs/deploy/job-scheduler.md @@ -14,7 +14,7 @@ The scheduler enables both cron-based and event-driven execution of dbt commands - Event-driven execution of dbt Cloud jobs that run based on the completion of another job ([trigger on job completion](/docs/deploy/deploy-jobs#trigger-on-job-completion)) - Event-driven execution of dbt Cloud CI jobs triggered by pull requests to the dbt repo ([merge jobs](/docs/deploy/merge-jobs)) - Event-driven execution of dbt Cloud jobs triggered by API -- Event-driven execution of dbt Cloud jobs manually triggered by a user to "Run Now" +- Event-driven execution of dbt Cloud jobs manually triggered by a user to **Run now** The scheduler handles various tasks including queuing jobs, creating temporary environments to run the dbt commands required for those jobs, providing logs for debugging and remediation, and storing dbt artifacts for direct consumption/ingestion by the Discovery API. From 1a05a186e17f464fb36e88f4a6e63aac75e6967b Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 17 Dec 2024 11:36:01 +0000 Subject: [PATCH 12/12] Update website/docs/docs/deploy/job-scheduler.md --- website/docs/docs/deploy/job-scheduler.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/deploy/job-scheduler.md b/website/docs/docs/deploy/job-scheduler.md index 1990832f7df..c5d1886879b 100644 --- a/website/docs/docs/deploy/job-scheduler.md +++ b/website/docs/docs/deploy/job-scheduler.md @@ -12,7 +12,7 @@ The scheduler enables both cron-based and event-driven execution of dbt commands - Cron-based execution of dbt Cloud jobs that run on a predetermined cadence - Event-driven execution of dbt Cloud jobs that run based on the completion of another job ([trigger on job completion](/docs/deploy/deploy-jobs#trigger-on-job-completion)) -- Event-driven execution of dbt Cloud CI jobs triggered by pull requests to the dbt repo ([merge jobs](/docs/deploy/merge-jobs)) +- Event-driven execution of dbt Cloud CI jobs triggered when a pull request is merged to the branch ([merge jobs](/docs/deploy/merge-jobs)) - Event-driven execution of dbt Cloud jobs triggered by API - Event-driven execution of dbt Cloud jobs manually triggered by a user to **Run now**