From cb232eec2de15d8c141ed3118b36825ec75b7d7d Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Fri, 3 Nov 2023 12:10:51 +0000 Subject: [PATCH 1/9] Update pre-hook-post-hook.md clarify transactions not supported --- .../resource-configs/pre-hook-post-hook.md | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/website/docs/reference/resource-configs/pre-hook-post-hook.md b/website/docs/reference/resource-configs/pre-hook-post-hook.md index 297d6975d6f..b4c779cc7bf 100644 --- a/website/docs/reference/resource-configs/pre-hook-post-hook.md +++ b/website/docs/reference/resource-configs/pre-hook-post-hook.md @@ -175,13 +175,16 @@ If multiple instances of any hooks are defined, dbt will run each hook using the ### Transaction behavior -If you're using an adapter that makes use of transactions (namely Postgres or Redshift), it's worth noting that by default hooks are executed inside of the same transaction as your model being created. +If you're using an adapter that uses transactions (namely Postgres or Redshift), it's worth noting that by default hooks are executed inside of the same transaction as your model being created. There may be occasions where you need to run these hooks _outside_ of a transaction, for example: -* You want to run a `VACUUM` in a `post-hook`, however this cannot be executed within a transaction ([Redshift docs](https://docs.aws.amazon.com/redshift/latest/dg/r_VACUUM_command.html#r_VACUUM_usage_notes)) -* You want to insert a record into an audit at the start of a run, and do not want that statement rolled back if the model creation fails. +* You want to run a `VACUUM` in a `post-hook`, however, this cannot be executed within a transaction ([Redshift docs](https://docs.aws.amazon.com/redshift/latest/dg/r_VACUUM_command.html#r_VACUUM_usage_notes)) +* You want to insert a record into an audit at the start of a run and do not want that statement rolled back if the model creation fails. -To achieve this, you can use one of the following syntaxes. (Note: You should NOT use this syntax if using a database where dbt does not use transactions by default, including Snowflake, BigQuery, and Spark/Databricks.) +To achieve this behavior, you can use one of the following syntaxes. (Important Note: Do not use this syntax if you are using a database where dbt does not use transactions by default. This includes databases like Snowflake, BigQuery, and Spark/Databricks.) + + + #### Config block: use the `before_begin` and `after_commit` helper macros @@ -200,6 +203,9 @@ select ... ``` + + + #### Config block: use a dictionary @@ -224,6 +230,10 @@ select ... + + + + #### `dbt_project.yml`: Use a dictionary @@ -242,3 +252,4 @@ models: ``` + From d43d129bcaba903ba96612b7c2cadc4d0b6ea263 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Fri, 3 Nov 2023 12:13:06 +0000 Subject: [PATCH 2/9] Update website/docs/reference/resource-configs/pre-hook-post-hook.md --- website/docs/reference/resource-configs/pre-hook-post-hook.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/reference/resource-configs/pre-hook-post-hook.md b/website/docs/reference/resource-configs/pre-hook-post-hook.md index b4c779cc7bf..0d8df81aa24 100644 --- a/website/docs/reference/resource-configs/pre-hook-post-hook.md +++ b/website/docs/reference/resource-configs/pre-hook-post-hook.md @@ -181,7 +181,7 @@ There may be occasions where you need to run these hooks _outside_ of a transact * You want to run a `VACUUM` in a `post-hook`, however, this cannot be executed within a transaction ([Redshift docs](https://docs.aws.amazon.com/redshift/latest/dg/r_VACUUM_command.html#r_VACUUM_usage_notes)) * You want to insert a record into an audit at the start of a run and do not want that statement rolled back if the model creation fails. -To achieve this behavior, you can use one of the following syntaxes. (Important Note: Do not use this syntax if you are using a database where dbt does not use transactions by default. This includes databases like Snowflake, BigQuery, and Spark/Databricks.) +To achieve this behavior, you can use one of the following syntaxes. (Important Note: Do not use this syntax if you are using a database where dbt does not use transactions by default. This includes databases like Snowflake, BigQuery, and Spark or Databricks.) From f465eb865b96524bf857d455d0170b45f2ff30a7 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Fri, 3 Nov 2023 12:13:28 +0000 Subject: [PATCH 3/9] Update website/docs/reference/resource-configs/pre-hook-post-hook.md --- website/docs/reference/resource-configs/pre-hook-post-hook.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/website/docs/reference/resource-configs/pre-hook-post-hook.md b/website/docs/reference/resource-configs/pre-hook-post-hook.md index 0d8df81aa24..f48b65b4ad6 100644 --- a/website/docs/reference/resource-configs/pre-hook-post-hook.md +++ b/website/docs/reference/resource-configs/pre-hook-post-hook.md @@ -181,7 +181,9 @@ There may be occasions where you need to run these hooks _outside_ of a transact * You want to run a `VACUUM` in a `post-hook`, however, this cannot be executed within a transaction ([Redshift docs](https://docs.aws.amazon.com/redshift/latest/dg/r_VACUUM_command.html#r_VACUUM_usage_notes)) * You want to insert a record into an audit at the start of a run and do not want that statement rolled back if the model creation fails. -To achieve this behavior, you can use one of the following syntaxes. (Important Note: Do not use this syntax if you are using a database where dbt does not use transactions by default. This includes databases like Snowflake, BigQuery, and Spark or Databricks.) +To achieve this behavior, you can use one of the following syntaxes. + +Important note: Do not use this syntax if you are using a database where dbt does not use transactions by default. This includes databases like Snowflake, BigQuery, and Spark or Databricks. From 35318a1c438e6363af7f5db57463a3b69001f1d4 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Fri, 3 Nov 2023 12:16:33 +0000 Subject: [PATCH 4/9] Update website/docs/reference/resource-configs/pre-hook-post-hook.md --- website/docs/reference/resource-configs/pre-hook-post-hook.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/reference/resource-configs/pre-hook-post-hook.md b/website/docs/reference/resource-configs/pre-hook-post-hook.md index f48b65b4ad6..a21955e1c06 100644 --- a/website/docs/reference/resource-configs/pre-hook-post-hook.md +++ b/website/docs/reference/resource-configs/pre-hook-post-hook.md @@ -186,7 +186,7 @@ To achieve this behavior, you can use one of the following syntaxes. Important note: Do not use this syntax if you are using a database where dbt does not use transactions by default. This includes databases like Snowflake, BigQuery, and Spark or Databricks. - + #### Config block: use the `before_begin` and `after_commit` helper macros From 88f463d2d6eba7ccd687ffe8e561aa1d59f429b6 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Fri, 3 Nov 2023 12:18:01 +0000 Subject: [PATCH 5/9] Update pre-hook-post-hook.md --- website/docs/reference/resource-configs/pre-hook-post-hook.md | 1 + 1 file changed, 1 insertion(+) diff --git a/website/docs/reference/resource-configs/pre-hook-post-hook.md b/website/docs/reference/resource-configs/pre-hook-post-hook.md index a21955e1c06..e3f03223ff5 100644 --- a/website/docs/reference/resource-configs/pre-hook-post-hook.md +++ b/website/docs/reference/resource-configs/pre-hook-post-hook.md @@ -255,3 +255,4 @@ models: + From a3eec5d318f757c21da8dc89a5bb93ed0939c1c3 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Fri, 3 Nov 2023 13:21:48 +0000 Subject: [PATCH 6/9] Update pre-hook-post-hook.md --- .../docs/reference/resource-configs/pre-hook-post-hook.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/website/docs/reference/resource-configs/pre-hook-post-hook.md b/website/docs/reference/resource-configs/pre-hook-post-hook.md index e3f03223ff5..0c631159d56 100644 --- a/website/docs/reference/resource-configs/pre-hook-post-hook.md +++ b/website/docs/reference/resource-configs/pre-hook-post-hook.md @@ -186,7 +186,7 @@ To achieve this behavior, you can use one of the following syntaxes. Important note: Do not use this syntax if you are using a database where dbt does not use transactions by default. This includes databases like Snowflake, BigQuery, and Spark or Databricks. - + #### Config block: use the `before_begin` and `after_commit` helper macros @@ -207,7 +207,7 @@ select ... - + #### Config block: use a dictionary @@ -234,7 +234,7 @@ select ... - + #### `dbt_project.yml`: Use a dictionary From efe8670147dac5ce10adac8f6084305a83a4bab7 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Fri, 3 Nov 2023 13:31:15 +0000 Subject: [PATCH 7/9] Update website/docs/reference/resource-configs/pre-hook-post-hook.md --- website/docs/reference/resource-configs/pre-hook-post-hook.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/reference/resource-configs/pre-hook-post-hook.md b/website/docs/reference/resource-configs/pre-hook-post-hook.md index 0c631159d56..fbc3e182eae 100644 --- a/website/docs/reference/resource-configs/pre-hook-post-hook.md +++ b/website/docs/reference/resource-configs/pre-hook-post-hook.md @@ -207,7 +207,7 @@ select ... - + #### Config block: use a dictionary From 6acedeafad07b303322eee3ae601bfc29fd9da80 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Fri, 3 Nov 2023 13:31:39 +0000 Subject: [PATCH 8/9] Update pre-hook-post-hook.md --- .../docs/reference/resource-configs/pre-hook-post-hook.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/website/docs/reference/resource-configs/pre-hook-post-hook.md b/website/docs/reference/resource-configs/pre-hook-post-hook.md index fbc3e182eae..16ef2e1685c 100644 --- a/website/docs/reference/resource-configs/pre-hook-post-hook.md +++ b/website/docs/reference/resource-configs/pre-hook-post-hook.md @@ -181,9 +181,8 @@ There may be occasions where you need to run these hooks _outside_ of a transact * You want to run a `VACUUM` in a `post-hook`, however, this cannot be executed within a transaction ([Redshift docs](https://docs.aws.amazon.com/redshift/latest/dg/r_VACUUM_command.html#r_VACUUM_usage_notes)) * You want to insert a record into an audit at the start of a run and do not want that statement rolled back if the model creation fails. -To achieve this behavior, you can use one of the following syntaxes. - -Important note: Do not use this syntax if you are using a database where dbt does not use transactions by default. This includes databases like Snowflake, BigQuery, and Spark or Databricks. +To achieve this behavior, you can use one of the following syntaxes: + - Important note: Do not use this syntax if you are using a database where dbt does not use transactions by default. This includes databases like Snowflake, BigQuery, and Spark or Databricks. From eb7aa1547856e8f7cff3b75c762655261931bf81 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 8 Nov 2023 11:14:43 +0000 Subject: [PATCH 9/9] Update website/docs/reference/resource-configs/pre-hook-post-hook.md Co-authored-by: Pat Kearns --- website/docs/reference/resource-configs/pre-hook-post-hook.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/reference/resource-configs/pre-hook-post-hook.md b/website/docs/reference/resource-configs/pre-hook-post-hook.md index 16ef2e1685c..de652bff088 100644 --- a/website/docs/reference/resource-configs/pre-hook-post-hook.md +++ b/website/docs/reference/resource-configs/pre-hook-post-hook.md @@ -182,7 +182,7 @@ There may be occasions where you need to run these hooks _outside_ of a transact * You want to insert a record into an audit at the start of a run and do not want that statement rolled back if the model creation fails. To achieve this behavior, you can use one of the following syntaxes: - - Important note: Do not use this syntax if you are using a database where dbt does not use transactions by default. This includes databases like Snowflake, BigQuery, and Spark or Databricks. + - Important note: Do not use this syntax if you are using a database where dbt does not support transactions. This includes databases like Snowflake, BigQuery, and Spark or Databricks.