Merge branch 'current' into runleonarun-patch-13

website/docs/best-practices/how-we-mesh/mesh-4-faqs.md

@@ -44,9 +44,9 @@ You can use model versions to:
 
 A [model access modifier](/docs/collaborate/govern/model-access) in dbt determines if a model is accessible as an input to other dbt models and projects. It specifies where a model can be referenced using [the `ref` function](/reference/dbt-jinja-functions/ref). There are three types of access modifiers:
 
-1. **Private:** A model with a private access modifier is only referenceable by models within the same group. This is intended for models that are implementation details and are meant to be used only within a specific group of related models.
-2. **Protected:** Models with a protected access modifier can be referenced by any other model within the same dbt project or when the project is installed as a package. This is the default setting for all models, ensuring backward compatibility, especially when groups are assigned to an existing set of models.
-3. **Public:** A public model can be referenced across different groups, packages, or projects. This is suitable for stable and mature models that serve as interfaces for other teams or projects.
+* **Private:** A model with a private access modifier is only referenceable by models within the same group. This is intended for models that are implementation details and are meant to be used only within a specific group of related models.
+* **Protected:** Models with a protected access modifier can be referenced by any other model within the same dbt project or when the project is installed as a package. This is the default setting for all models, ensuring backward compatibility, especially when groups are assigned to an existing set of models.
+* **Public:** A public model can be referenced across different groups, packages, or projects. This is suitable for stable and mature models that serve as interfaces for other teams or projects.
 
 </detailsToggle>
 
@@ -208,12 +208,12 @@ First things first: access to underlying data is always defined and enforced by
 
 [Model access](/docs/collaborate/govern/model-access) defines where models can be referenced. It also informs the discoverability of those projects within dbt Explorer. Model `access` is defined in code, just like any other model configuration (`materialized`, `tags`, etc).
 
-**Public:** Models with `public` access can be referenced everywhere. These are the “data products” of your organization.
+* **Public:** Models with `public` access can be referenced everywhere. These are the “data products” of your organization.
 
-**Protected:** Models with `protected` access can only be referenced within the same project. This is the default level of model access. 
+* **Protected:** Models with `protected` access can only be referenced within the same project. This is the default level of model access. 
 We are discussing a future extension to `protected` models to allow for their reference in _specific_ downstream projects. Please read [the GitHub issue](https://github.com/dbt-labs/dbt-core/issues/9340), and upvote/comment if you’re interested in this use case.
 
-**Private:** Model `groups` enable more-granular control over where `private` models can be referenced. By defining a group, and configuring models to belong to that group, you can restrict other models (not in the same group) from referencing any `private` models the group contains. Groups also provide a standard mechanism for defining the `owner` of all resources it contains.
+* **Private:** Model `groups` enable more-granular control over where `private` models can be referenced. By defining a group, and configuring models to belong to that group, you can restrict other models (not in the same group) from referencing any `private` models the group contains. Groups also provide a standard mechanism for defining the `owner` of all resources it contains.
 
 Within dbt Explorer, `public` models are discoverable for every user in the dbt Cloud account — every public model is listed in the “multi-project” view. By contrast, `protected` and `private` models in a project are visible only to users who have access to that project (including read-only access).
 

website/docs/docs/build/incremental-models.md

@@ -236,7 +236,7 @@ Instead, whenever the logic of your incremental changes, execute a full-refresh
 
 ## About `incremental_strategy`
 
-There are various ways (strategies) to implement the concept of an incremental materializations. The value of each strategy depends on:
+There are various ways (strategies) to implement the concept of incremental materializations. The value of each strategy depends on:
 
 * the volume of data,
 * the reliability of your `unique_key`, and
@@ -450,5 +450,129 @@ The syntax depends on how you configure your `incremental_strategy`:
 
 </VersionBlock>
 
+### Built-in strategies
+
+Before diving into [custom strategies](#custom-strategies), it's important to understand the built-in incremental strategies in dbt and their corresponding macros:
+
+| `incremental_strategy` | Corresponding macro                    |
+|------------------------|----------------------------------------|
+| `append`               | `get_incremental_append_sql`           |
+| `delete+insert`        | `get_incremental_delete_insert_sql`    |
+| `merge`                | `get_incremental_merge_sql`            |
+| `insert_overwrite`     | `get_incremental_insert_overwrite_sql` |
+
+
+For example, a built-in strategy for the `append` can be defined and used with the following files:
+
+<File name='macros/append.sql'>
+
+```sql
+{% macro get_incremental_append_sql(arg_dict) %}
+
+  {% do return(some_custom_macro_with_sql(arg_dict["target_relation"], arg_dict["temp_relation"], arg_dict["unique_key"], arg_dict["dest_columns"], arg_dict["incremental_predicates"])) %}
+
+{% endmacro %}
+
+
+{% macro some_custom_macro_with_sql(target_relation, temp_relation, unique_key, dest_columns, incremental_predicates) %}
+
+    {%- set dest_cols_csv = get_quoted_csv(dest_columns | map(attribute="name")) -%}
+
+    insert into {{ target_relation }} ({{ dest_cols_csv }})
+    (
+        select {{ dest_cols_csv }}
+        from {{ temp_relation }}
+    )
+
+{% endmacro %}
+```
+</File>
+
+Define a model models/my_model.sql:
+
+```sql
+{{ config(
+    materialized="incremental",
+    incremental_strategy="append",
+) }}
+
+select * from {{ ref("some_model") }}
+```
+
+### Custom strategies
+
+<VersionBlock lastVersion="1.1">
+
+Custom incremental strategies can be defined beginning in dbt v1.2.
+
+</VersionBlock>
+
+<VersionBlock firstVersion="1.2">
+
+As an easier alternative to [creating an entirely new materialization](/guides/create-new-materializations), users can define and use their own "custom" user-defined incremental strategies by:
+
+1. defining a macro named `get_incremental_STRATEGY_sql`. Note that `STRATEGY` is a placeholder and you should replace it with the name of your custom incremental strategy.
+2. configuring `incremental_strategy: STRATEGY` within an incremental model
+
+dbt won't validate user-defined strategies, it will just look for the macro by that name, and raise an error if it can't find one.
+
+For example, a user-defined strategy named `insert_only` can be defined and used with the following files:
+
+<File name='macros/my_custom_strategies.sql'>
+
+```sql
+{% macro get_incremental_insert_only_sql(arg_dict) %}
+
+  {% do return(some_custom_macro_with_sql(arg_dict["target_relation"], arg_dict["temp_relation"], arg_dict["unique_key"], arg_dict["dest_columns"], arg_dict["incremental_predicates"])) %}
+
+{% endmacro %}
+
+
+{% macro some_custom_macro_with_sql(target_relation, temp_relation, unique_key, dest_columns, incremental_predicates) %}
+
+    {%- set dest_cols_csv = get_quoted_csv(dest_columns | map(attribute="name")) -%}
+
+    insert into {{ target_relation }} ({{ dest_cols_csv }})
+    (
+        select {{ dest_cols_csv }}
+        from {{ temp_relation }}
+    )
+
+{% endmacro %}
+```
+
+</File>
+
+<File name='models/my_model.sql'>
+
+```sql
+{{ config(
+    materialized="incremental",
+    incremental_strategy="insert_only",
+    ...
+) }}
+
+...
+```
+
+</File>
+
+### Custom strategies from a package
+
+To use the `merge_null_safe` custom incremental strategy from the `example` package:
+- [Install the package](/docs/build/packages#how-do-i-add-a-package-to-my-project)
+- Then add the following macro to your project:
+
+<File name='macros/my_custom_strategies.sql'>
+
+```sql
+{% macro get_incremental_merge_null_safe_sql(arg_dict) %}
+    {% do return(example.get_incremental_merge_null_safe_sql(arg_dict)) %}
+{% endmacro %}
+```
+
+</File>
+</VersionBlock>
+
 <Snippet path="discourse-help-feed-header" />
 <DiscourseHelpFeed tags="incremental"/>

website/docs/guides/manual-install-qs.md

@@ -70,7 +70,7 @@ $ pwd
 <Lightbox src="/img/starter-project-dbt-cli.png" title="The starter project in a code editor" />
 </div>
 
-6. Update the following values in the `dbt_project.yml` file:
+6. dbt provides the following values in the `dbt_project.yml` file:
 
 <File name='dbt_project.yml'>
 
@@ -92,7 +92,7 @@ models:
 
 ## Connect to BigQuery
 
-When developing locally, dbt connects to your <Term id="data-warehouse" /> using a [profile](/docs/core/connect-data-platform/connection-profiles), which is a YAML file with all the connection details to your warehouse.
+When developing locally, dbt connects to your <Term id="data-warehouse" /> using a [profile](/docs/core/connect-data-platform/connection-profiles), which is a YAML file with all the connection details to your warehouse. 
 
 1. Create a file in the `~/.dbt/` directory named `profiles.yml`.
 2. Move your BigQuery keyfile into this directory.