Skip to content

Commit

Permalink
Merge branch 'current' into mirnawong1-patch-28
Browse files Browse the repository at this point in the history
  • Loading branch information
runleonarun authored Dec 24, 2024
2 parents d675e8f + b214a24 commit ca11bf5
Show file tree
Hide file tree
Showing 17 changed files with 48 additions and 46 deletions.
1 change: 0 additions & 1 deletion website/docs/docs/build/environment-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,6 @@ dbt Cloud has a number of pre-defined variables built in. Variables are set auto
The following environment variable is set automatically for the dbt Cloud IDE:

- `DBT_CLOUD_GIT_BRANCH` — Provides the development Git branch name in the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud).
- Available in dbt v1.6 and later.
- The variable changes when the branch is changed.
- Doesn't require restarting the IDE after a branch change.
- Currently not available in the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation).
Expand Down
2 changes: 1 addition & 1 deletion website/docs/docs/build/incremental-microbatch.md
Original file line number Diff line number Diff line change
Expand Up @@ -232,7 +232,7 @@ from {{ source('sales', 'transactions') }}

### Full refresh

As a best practice, we recommend configuring `full_refresh: False` on microbatch models so that they ignore invocations with the `--full-refresh` flag. If you need to reprocess historical data, do so with a targeted backfill that specifies explicit start and end dates.
As a best practice, we recommend [configuring `full_refresh: false`](/reference/resource-configs/full_refresh) on microbatch models so that they ignore invocations with the `--full-refresh` flag. If you need to reprocess historical data, do so with a targeted backfill that specifies explicit start and end dates.

## Usage

Expand Down
5 changes: 2 additions & 3 deletions website/docs/docs/build/unit-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,10 @@ keywords:
- unit test, unit tests, unit testing, dag
---

:::note
<VersionCallout version="1.8" />


Unit testing functionality is available in [dbt Cloud Release Tracks](/docs/dbt-versions/cloud-release-tracks) or dbt Core v1.8+

:::

Historically, dbt's test coverage was confined to [“data” tests](/docs/build/data-tests), assessing the quality of input data or resulting datasets' structure. However, these tests could only be executed _after_ building a model.

Expand Down
16 changes: 12 additions & 4 deletions website/docs/docs/deploy/model-notifications.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,21 +38,29 @@ groups:
- name: finance
owner:
# Email is required to receive model-level notifications, additional properties are also allowed.
name: "Finance Team"
description: "Models related to the finance department"
name: "Finance team"
email: [email protected]
favorite_food: donuts

- name: marketing
owner:
name: "Marketing Team"
description: "Models related to the marketing department"
name: "Marketing team"
email: [email protected]
favorite_food: jaffles

- name: docs
owner:
name: "Documentation team"
email: [email protected]
favorite_food: pizza
```
</File>
:::tip
The `owner` key is flexible and accepts arbitrary inputs in addition to the required `email` field. For example, you could include a custom field like `favorite_food` to add context about the team.
:::

## Attach groups to models

Attach groups to models as you would any other config, in either the `dbt_project.yml` or `whatever.yml` files. For example:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Running with dbt=1.0.0
Found 13 models, 2 tests, 1 archives, 0 analyses, 204 macros, 2 operations....
```

:::info dbt Cloud release tracks
:::note dbt Cloud release tracks
<Snippet path="_config-dbt-version-check" />

:::
2 changes: 1 addition & 1 deletion website/docs/reference/resource-configs/batch_size.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ description: "dbt uses `batch_size` to determine how large batches are when runn
datatype: hour | day | month | year
---

Available in the [dbt Cloud "Latest" release track](/docs/dbt-versions/cloud-release-tracks) and dbt Core v1.9 and higher.
<VersionCallout version="1.9" />

## Definition

Expand Down
2 changes: 1 addition & 1 deletion website/docs/reference/resource-configs/begin.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ description: "dbt uses `begin` to determine when a microbatch incremental model
datatype: string
---

Available in the [dbt Cloud "Latest" release track](/docs/dbt-versions/cloud-release-tracks) and dbt Core v1.9 and higher.
<VersionCallout version="1.9" />

## Definition

Expand Down
4 changes: 2 additions & 2 deletions website/docs/reference/resource-configs/databricks-configs.md
Original file line number Diff line number Diff line change
Expand Up @@ -437,8 +437,8 @@ Beginning with 1.9, `merge` behavior can be modified with the following addition
- `skip_not_matched_step`: If set to `true`, the 'not matched' clause will not be included.
- `matched_condition`: Condition to apply to the `WHEN MATCHED` clause. You should use the `target_alias` and `source_alias` to write a conditional expression, such as `DBT_INTERNAL_DEST.col1 = hash(DBT_INTERNAL_SOURCE.col2, DBT_INTERNAL_SOURCE.col3)`. This condition further restricts the matched set of rows.
- `not_matched_condition`: Condition to apply to the `WHEN NOT MATCHED [BY TARGET]` clause. This condition further restricts the set of rows in the target that do not match the source that will be inserted into the merged table.
- `not_matched_by_source_condition`: Condition to apply to the further filter `WHEN NOT MATCHED BY SOURCE` clause. Only used in conjunction with `not_matched_by_source_action: delete`.
- `not_matched_by_source_action`: If set to `delete`, a `DELETE` clause is added to the merge statement for `WHEN NOT MATCHED BY SOURCE`.
- `not_matched_by_source_condition`: Condition to apply to the further filter `WHEN NOT MATCHED BY SOURCE` clause. Only used in conjunction with `not_matched_by_source_action`.
- `not_matched_by_source_action`: The action to apply when the condition is met. Configure as an expression. For example: `not_matched_by_source_action: "update set t.attr1 = 'deleted', t.tech_change_ts = current_timestamp()"`.
- `merge_with_schema_evolution`: If set to `true`, the merge statement includes the `WITH SCHEMA EVOLUTION` clause.

For more details on the meaning of each merge clause, please see [the Databricks documentation](https://docs.databricks.com/en/sql/language-manual/delta-merge-into.html).
Expand Down
2 changes: 1 addition & 1 deletion website/docs/reference/resource-configs/delimiter.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ datatype: <string>
default_value: ","
---

Supported in v1.7 and higher.
<VersionCallout version="1.7" />

## Definition

Expand Down
2 changes: 1 addition & 1 deletion website/docs/reference/resource-configs/event-time.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ description: "dbt uses event_time to understand when an event occurred. When def
datatype: string
---

Available in [the dbt Cloud "Latest" release track](/docs/dbt-versions/cloud-release-tracks) and dbt Core v1.9 and higher.
<VersionCallout version="1.9" />

<Tabs>
<TabItem value="model" label="Models">
Expand Down
35 changes: 20 additions & 15 deletions website/docs/reference/resource-configs/full_refresh.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ description: "Setting the full_refresh config to false prevents a model or seed
datatype: boolean
---

The `full_refresh` config allows you to control whether a resource will always or never perform a full-refresh. This config overrides the `--full-refresh` command-line flag.

<Tabs
defaultValue="models"
values={[
Expand All @@ -19,8 +21,7 @@ datatype: boolean
```yml
models:
[<resource-path>](/reference/resource-configs/resource-path):
+full_refresh: false

+full_refresh: false | true
```
</File>
Expand All @@ -30,17 +31,14 @@ models:
```sql

{{ config(
full_refresh = false
full_refresh = false | true
) }}

select ...

```

</File>

The configured model(s) will not full-refresh when `dbt run --full-refresh` is invoked.

</TabItem>

<TabItem value="seeds">
Expand All @@ -50,27 +48,34 @@ The configured model(s) will not full-refresh when `dbt run --full-refresh` is i
```yml
seeds:
[<resource-path>](/reference/resource-configs/resource-path):
+full_refresh: false
+full_refresh: false | true

```

</File>

The configured seed(s) will not full-refresh when `dbt seed --full-refresh` is invoked.

</TabItem>

</Tabs>

- If `full_refresh:true` &mdash; the configured resources(s) will full-refresh when `dbt run --full-refresh` is invoked.
- If `full_refresh:false` &mdash; the configured resources(s) will _not_ full-refresh when `dbt run --full-refresh` is invoked.


## Description
Optionally set a resource to always or never full-refresh.
- If specified as `true` or `false`, the
`full_refresh` config will take precedence over the presence or absence of the `--full-refresh` flag.
- If the `full_refresh` config is `none` or omitted, the resource will use the value of the `--full-refresh` flag.

**Note:** The `--full-refresh` flag also supports a short name, `-f`.
The `full_refresh` config allows you to optionally configure whether a resource will always or never perform a full-refresh. This config is an override for the `--full-refresh` command line flag used when running dbt commands.


| `full_refresh` value | Behavior |
| ---------------------------- | -------- |
| `true` | The resource always full-refreshes, regardless of the presence or absence of the `--full-refresh` flag. |
| `false` | The resource never full-refreshes, even if the `--full-refresh` flag is provided. |
| `none` or omitted | The resource follows the behavior of the `--full-refresh` flag. If the flag is used, the resource will full-refresh; otherwise, it won't. |

This logic is encoded in the [`should_full_refresh()`](https://github.com/dbt-labs/dbt-adapters/blob/60005a0a2bd33b61cb65a591bc1604b1b3fd25d5/dbt/include/global_project/macros/materializations/configs.sql) macro.
#### Note
- The `--full-refresh` flag also supports a short name, `-f`.
- The [`should_full_refresh()`](https://github.com/dbt-labs/dbt-adapters/blob/60005a0a2bd33b61cb65a591bc1604b1b3fd25d5/dbt/include/global_project/macros/materializations/configs.sql) macro has logic encoded.

## Usage

Expand Down
3 changes: 1 addition & 2 deletions website/docs/reference/resource-configs/hard-deletes.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,7 @@ id: "hard-deletes"
sidebar_label: "hard_deletes"
---

Available from dbt v1.9 or with [dbt Cloud "Latest" release track](/docs/dbt-versions/cloud-release-tracks).

<VersionCallout version="1.9" />

<File name='snapshots/schema.yml'>

Expand Down
3 changes: 1 addition & 2 deletions website/docs/reference/resource-configs/lookback.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,7 @@ description: "dbt uses `lookback` to detrmine how many 'batches' of `batch_size`
datatype: int
---

Available in the [dbt Cloud "Latest" release track](/docs/dbt-versions/cloud-release-tracks) and dbt Core v1.9 and higher.

<VersionCallout version="1.9" />
## Definition

Set the `lookback` to an integer greater than or equal to zero. The default value is `1`. You can configure `lookback` for a [model](/docs/build/models) in your `dbt_project.yml` file, property YAML file, or config block.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ default_value: {"dbt_valid_from": "dbt_valid_from", "dbt_valid_to": "dbt_valid_t
id: "snapshot_meta_column_names"
---

Available in dbt Core v1.9+. Select v1.9 or newer from the version dropdown to view the configs. Try it now in the [dbt Cloud "Latest" release track](/docs/dbt-versions/cloud-release-tracks).
<VersionCallout version="1.9" />

<File name='snapshots/schema.yml'>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,7 @@ datatype: model_name
description: "Learn about concurrent_batches in dbt."
---

:::note

Available in dbt Core v1.9+ or the [dbt Cloud "Latest" release tracks](/docs/dbt-versions/cloud-release-tracks).

:::
<VersionCallout version="1.9" />

<Tabs>
<TabItem value="Project file">
Expand Down
5 changes: 1 addition & 4 deletions website/docs/reference/resource-properties/unit-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,8 @@ resource_types: [models]
datatype: test
---

:::note
<VersionCallout version="1.8" />

This functionality is available in dbt Core v1.8+ and [dbt Cloud release tracks](/docs/dbt-versions/cloud-release-tracks).

:::

Unit tests validate your SQL modeling logic on a small set of static inputs before you materialize your full model in production. They support a test-driven development approach, improving both the efficiency of developers and reliability of code.

Expand Down
2 changes: 1 addition & 1 deletion website/src/components/versionCallout/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ const VersionCallout = ({ version }) => {
Available from dbt v{version} or with the{' '}
<a href="/docs/dbt-versions/cloud-release-tracks">
dbt Cloud "Latest" release track
</a>{' '}.
</a>{''}.
</span>
</Admonition>
</div>
Expand Down

0 comments on commit ca11bf5

Please sign in to comment.