From 7b07a591bcc61084fd9b2c9310b4deff14de3f35 Mon Sep 17 00:00:00 2001 From: Alex Coleman Date: Wed, 11 Dec 2024 15:42:55 +0000 Subject: [PATCH 1/9] Correct bind mount for profiles.yml Corrects the bind mount for profiles.yml which previously mounted the host profiles.yml file to /root/.dbt. --- website/docs/docs/core/docker-install.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/core/docker-install.md b/website/docs/docs/core/docker-install.md index 5ce033c7996..3cf066e2231 100644 --- a/website/docs/docs/core/docker-install.md +++ b/website/docs/docs/core/docker-install.md @@ -34,7 +34,7 @@ The `ENTRYPOINT` for dbt Docker images is the command `dbt`. You can bind-mount docker run \ --network=host \ --mount type=bind,source=path/to/project,target=/usr/app \ ---mount type=bind,source=path/to/profiles.yml,target=/root/.dbt/ \ +--mount type=bind,source=path/to/profiles.yml,target=/root/.dbt/profiles.yml \ \ ls ``` From 0c9a5f224a850d584b0d67adeeca58a7a94b9b27 Mon Sep 17 00:00:00 2001 From: rcg798 <5733407+rcg798@users.noreply.github.com> Date: Mon, 16 Dec 2024 10:05:42 -0800 Subject: [PATCH 2/9] Update job-scheduler.md Coming newly into the orchestration team, I am updating the initial description of the scheduler types to better align with how users may view the product today: 1. we refer to merge jobs as merge jobs (and it is an explicit category in the UI) so I think it would be useful to call out that term in the beginning, so people associate that type of job with its name and 2. another event-based job is job completion (another explicit category in the UI) and I couldn't tell how that fit into the initial description, so added that as well. Feel free to revise, just wanted to flag those two points! --- website/docs/docs/deploy/job-scheduler.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/website/docs/docs/deploy/job-scheduler.md b/website/docs/docs/deploy/job-scheduler.md index 7d45fddc3f6..fb351703624 100644 --- a/website/docs/docs/deploy/job-scheduler.md +++ b/website/docs/docs/deploy/job-scheduler.md @@ -11,7 +11,8 @@ The job scheduler is the backbone of running jobs in dbt Cloud, bringing power a The scheduler enables both cron-based and event-driven execution of dbt commands in the user’s data platform. Specifically, it handles: - Cron-based execution of dbt Cloud jobs that run on a predetermined cadence -- Event-driven execution of dbt Cloud CI jobs triggered by pull requests to the dbt repo +- Event-driven execution of dbt Cloud jobs that run based on the completion of another job +- Event-driven execution of dbt Cloud CI jobs triggered by pull requests to the dbt repo (e.g. "merge jobs") - Event-driven execution of dbt Cloud jobs triggered by API - Event-driven execution of dbt Cloud jobs manually triggered by a user to "Run Now" From a7e7e1bc9f6d9a850f4c13c790106c516d7ffa53 Mon Sep 17 00:00:00 2001 From: bethanyhipple-dbtlabs <108838013+bethanyhipple-dbtlabs@users.noreply.github.com> Date: Mon, 16 Dec 2024 14:58:14 -0800 Subject: [PATCH 3/9] Remove extra fields in create table statements The CSV files for orders and payment don't have _etl_loaded_at and batched_at respectively. So removing these fields from the create table statements. --- website/docs/guides/redshift-qs.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/website/docs/guides/redshift-qs.md b/website/docs/guides/redshift-qs.md index 83fafad1d12..8990c4db925 100644 --- a/website/docs/guides/redshift-qs.md +++ b/website/docs/guides/redshift-qs.md @@ -118,8 +118,7 @@ Now we are going to load our sample data into the S3 bucket that our Cloudformat id integer, user_id integer, order_date date, - status varchar(50), - _etl_loaded_at timestamp default current_timestamp + status varchar(50) ); create table stripe.payment( @@ -128,8 +127,7 @@ Now we are going to load our sample data into the S3 bucket that our Cloudformat paymentmethod varchar(50), status varchar(50), amount integer, - created date, - _batched_at timestamp default current_timestamp + created date ); ``` From 0fd24b183b9640f88739e337dfd0c31a7b10dfc2 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 17 Dec 2024 10:20:15 +0000 Subject: [PATCH 4/9] Update docker-install.md both options work as confirmed by @jeremyyeo --- website/docs/docs/core/docker-install.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/website/docs/docs/core/docker-install.md b/website/docs/docs/core/docker-install.md index 3cf066e2231..fc3caa42a03 100644 --- a/website/docs/docs/core/docker-install.md +++ b/website/docs/docs/core/docker-install.md @@ -30,6 +30,7 @@ docker pull ghcr.io/dbt-labs/: ### Running a dbt Docker image in a container The `ENTRYPOINT` for dbt Docker images is the command `dbt`. You can bind-mount your project to `/usr/app` and use dbt as normal: + ``` docker run \ --network=host \ @@ -39,6 +40,17 @@ docker run \ ls ``` +Or + +``` +docker run \ +--network=host \ +--mount type=bind,source=path/to/project,target=/usr/app \ +--mount type=bind,source=path/to/profiles.yml.dbt,target=/root/.dbt/ \ + \ +ls +``` + Notes: * Bind-mount sources _must_ be an absolute path * You may need to make adjustments to the docker networking setting depending on the specifics of your data warehouse or database host. From 6b18573cadb772d466f1ae43c713e3fb38c2e800 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 17 Dec 2024 11:24:10 +0000 Subject: [PATCH 5/9] Update website/docs/docs/deploy/job-scheduler.md --- website/docs/docs/deploy/job-scheduler.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/deploy/job-scheduler.md b/website/docs/docs/deploy/job-scheduler.md index fb351703624..e8a28bbfb27 100644 --- a/website/docs/docs/deploy/job-scheduler.md +++ b/website/docs/docs/deploy/job-scheduler.md @@ -11,7 +11,7 @@ The job scheduler is the backbone of running jobs in dbt Cloud, bringing power a The scheduler enables both cron-based and event-driven execution of dbt commands in the user’s data platform. Specifically, it handles: - Cron-based execution of dbt Cloud jobs that run on a predetermined cadence -- Event-driven execution of dbt Cloud jobs that run based on the completion of another job +- Event-driven execution of dbt Cloud jobs that [run based on the completion of another job ([trigger on job completion](/docs/deploy/deploy-jobs#trigger-on-job-completion)) - Event-driven execution of dbt Cloud CI jobs triggered by pull requests to the dbt repo (e.g. "merge jobs") - Event-driven execution of dbt Cloud jobs triggered by API - Event-driven execution of dbt Cloud jobs manually triggered by a user to "Run Now" From d079ab41aaf7ceb0b7cb7b925bdbb70ef13000d9 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 17 Dec 2024 11:24:46 +0000 Subject: [PATCH 6/9] Update website/docs/docs/deploy/job-scheduler.md --- website/docs/docs/deploy/job-scheduler.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/deploy/job-scheduler.md b/website/docs/docs/deploy/job-scheduler.md index e8a28bbfb27..38a98d2f92e 100644 --- a/website/docs/docs/deploy/job-scheduler.md +++ b/website/docs/docs/deploy/job-scheduler.md @@ -12,7 +12,7 @@ The scheduler enables both cron-based and event-driven execution of dbt commands - Cron-based execution of dbt Cloud jobs that run on a predetermined cadence - Event-driven execution of dbt Cloud jobs that [run based on the completion of another job ([trigger on job completion](/docs/deploy/deploy-jobs#trigger-on-job-completion)) -- Event-driven execution of dbt Cloud CI jobs triggered by pull requests to the dbt repo (e.g. "merge jobs") +- Event-driven execution of dbt Cloud CI jobs triggered by pull requests to the dbt repo ([merge jobs](/docs/deploy/merge-jobs)) - Event-driven execution of dbt Cloud jobs triggered by API - Event-driven execution of dbt Cloud jobs manually triggered by a user to "Run Now" From f99d323aa52885f1c77b6c83655b4a37bc200596 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 17 Dec 2024 11:24:59 +0000 Subject: [PATCH 7/9] Update website/docs/docs/deploy/job-scheduler.md --- website/docs/docs/deploy/job-scheduler.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/deploy/job-scheduler.md b/website/docs/docs/deploy/job-scheduler.md index 38a98d2f92e..f3f5706e93f 100644 --- a/website/docs/docs/deploy/job-scheduler.md +++ b/website/docs/docs/deploy/job-scheduler.md @@ -11,7 +11,7 @@ The job scheduler is the backbone of running jobs in dbt Cloud, bringing power a The scheduler enables both cron-based and event-driven execution of dbt commands in the user’s data platform. Specifically, it handles: - Cron-based execution of dbt Cloud jobs that run on a predetermined cadence -- Event-driven execution of dbt Cloud jobs that [run based on the completion of another job ([trigger on job completion](/docs/deploy/deploy-jobs#trigger-on-job-completion)) +- Event-driven execution of dbt Cloud jobs that run based on the completion of another job ([trigger on job completion](/docs/deploy/deploy-jobs#trigger-on-job-completion)) - Event-driven execution of dbt Cloud CI jobs triggered by pull requests to the dbt repo ([merge jobs](/docs/deploy/merge-jobs)) - Event-driven execution of dbt Cloud jobs triggered by API - Event-driven execution of dbt Cloud jobs manually triggered by a user to "Run Now" From 4f4bdb985ae8bfde3be0ba28482e87df9697b80b Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 17 Dec 2024 11:25:39 +0000 Subject: [PATCH 8/9] Update website/docs/docs/deploy/job-scheduler.md --- website/docs/docs/deploy/job-scheduler.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/deploy/job-scheduler.md b/website/docs/docs/deploy/job-scheduler.md index f3f5706e93f..1990832f7df 100644 --- a/website/docs/docs/deploy/job-scheduler.md +++ b/website/docs/docs/deploy/job-scheduler.md @@ -14,7 +14,7 @@ The scheduler enables both cron-based and event-driven execution of dbt commands - Event-driven execution of dbt Cloud jobs that run based on the completion of another job ([trigger on job completion](/docs/deploy/deploy-jobs#trigger-on-job-completion)) - Event-driven execution of dbt Cloud CI jobs triggered by pull requests to the dbt repo ([merge jobs](/docs/deploy/merge-jobs)) - Event-driven execution of dbt Cloud jobs triggered by API -- Event-driven execution of dbt Cloud jobs manually triggered by a user to "Run Now" +- Event-driven execution of dbt Cloud jobs manually triggered by a user to **Run now** The scheduler handles various tasks including queuing jobs, creating temporary environments to run the dbt commands required for those jobs, providing logs for debugging and remediation, and storing dbt artifacts for direct consumption/ingestion by the Discovery API. From 1a05a186e17f464fb36e88f4a6e63aac75e6967b Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 17 Dec 2024 11:36:01 +0000 Subject: [PATCH 9/9] Update website/docs/docs/deploy/job-scheduler.md --- website/docs/docs/deploy/job-scheduler.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/deploy/job-scheduler.md b/website/docs/docs/deploy/job-scheduler.md index 1990832f7df..c5d1886879b 100644 --- a/website/docs/docs/deploy/job-scheduler.md +++ b/website/docs/docs/deploy/job-scheduler.md @@ -12,7 +12,7 @@ The scheduler enables both cron-based and event-driven execution of dbt commands - Cron-based execution of dbt Cloud jobs that run on a predetermined cadence - Event-driven execution of dbt Cloud jobs that run based on the completion of another job ([trigger on job completion](/docs/deploy/deploy-jobs#trigger-on-job-completion)) -- Event-driven execution of dbt Cloud CI jobs triggered by pull requests to the dbt repo ([merge jobs](/docs/deploy/merge-jobs)) +- Event-driven execution of dbt Cloud CI jobs triggered when a pull request is merged to the branch ([merge jobs](/docs/deploy/merge-jobs)) - Event-driven execution of dbt Cloud jobs triggered by API - Event-driven execution of dbt Cloud jobs manually triggered by a user to **Run now**