Skip to content

Commit

Permalink
Adding store_failures_as (#4270)
Browse files Browse the repository at this point in the history
## What are you changing in this pull request and why?

Closes: #4246

Adding a page for resource config `store_failures_as`

## Checklist

- [x] Add versioning components, as described in [Versioning
Docs](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#versioning-entire-pages)
- [x] Add a note to the prerelease version [Migration
Guide](https://github.com/dbt-labs/docs.getdbt.com/tree/current/website/docs/guides/migration/versions)

- [x] Review the [Content style
guide](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/content-style-guide.md)
and [About
versioning](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#adding-a-new-version)
so my content adheres to these guidelines.
- [x] Add a checklist item for anything that needs to happen before this
PR is merged, such as "needs technical review" or "change base branch."

Adding new pages (delete if not applicable):
- [x] Add page to `website/sidebars.js`
- [x] Provide a unique filename for the new page
  • Loading branch information
matthewshaver authored Oct 12, 2023
2 parents 70d50d4 + dba4fe7 commit 0eef471
Show file tree
Hide file tree
Showing 5 changed files with 83 additions and 2 deletions.
4 changes: 4 additions & 0 deletions website/dbt-versions.js
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,10 @@ exports.versions = [
]

exports.versionedPages = [
{
"page": "reference/resource-configs/store_failures_as",
"firstVersion": "1.7",
},
{
"page": "docs/build/build-metrics-intro",
"firstVersion": "1.6",
Expand Down
2 changes: 1 addition & 1 deletion website/docs/docs/build/tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -241,7 +241,7 @@ where {{ column_name }} is null

## Storing test failures

Normally, a test query will calculate failures as part of its execution. If you set the optional `--store-failures` flag or [`store_failures` config](/reference/resource-configs/store_failures), dbt will first save the results of a test query to a table in the database, and then query that table to calculate the number of failures.
Normally, a test query will calculate failures as part of its execution. If you set the optional `--store-failures` flag, the [`store_failures`](/reference/resource-configs/store_failures), or the [`store_failures_as`](/reference/resource-configs/store_failures_as) configs, dbt will first save the results of a test query to a table in the database, and then query that table to calculate the number of failures.

This workflow allows you to query and examine failing records much more quickly in development:

Expand Down
2 changes: 1 addition & 1 deletion website/docs/reference/resource-configs/store_failures.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ resource_types: [tests]
datatype: boolean
---

The configured test(s) will store their failures when `dbt test --store-failures` is invoked.
The configured test(s) will store their failures when `dbt test --store-failures` is invoked. If you set this configuration as `false` but [`store_failures_as`](/reference/resource-configs/store_failures_as) is configured, it will be overriden.

## Description
Optionally set a test to always or never store its failures in the database.
Expand Down
76 changes: 76 additions & 0 deletions website/docs/reference/resource-configs/store_failures_as.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
---
resource_types: [tests]
id: "store_failures_as"
---

For the `test` resource type, `store_failures_as` is an optional config that specifies how test failures should be stored in the database. If [`store_failures`](/reference/resource-configs/store_failures) is also configured, `store_failures_as` takes precedence.

The three supported values are:

- `ephemeral` — nothing stored in the database (default)
- `table` — test failures stored as a database table
- `view` — test failures stored as a database view

You can configure it in all the same places as `store_failures`, including singular tests (.sql files), generic tests (.yml files), and dbt_project.yml.

### Examples

#### Singular test

[Singular test](https://docs.getdbt.com/docs/build/tests#singular-tests) in `tests/singular/check_something.sql` file

```sql
{{ config(store_failures_as="table") }}

-- custom singular test
select 1 as id
where 1=0
```

#### Generic test

[Generic tests](https://docs.getdbt.com/docs/build/tests#generic-tests) in `models/_models.yml` file

```yaml
models:
- name: my_model
columns:
- name: id
tests:
- not_null:
config:
store_failures_as: view
- unique:
config:
store_failures_as: ephemeral
```
#### Project level
Config in `dbt_project.yml`

```yaml
name: "my_project"
version: "1.0.0"
config-version: 2
profile: "sandcastle"
tests:
my_project:
+store_failures_as: table
my_subfolder_1:
+store_failures_as: view
my_subfolder_2:
+store_failures_as: ephemeral
```

### "Clobbering" configs

As with most other configurations, `store_failures_as` is "clobbered" when applied hierarchically. Whenever a more specific value is available, it will completely replace the less specific value.

Additional resources:

- [Test configurations](/reference/test-configs#related-documentation)
- [Test-specific configurations](/reference/test-configs#test-specific-configurations)
- [Configuring directories of models in dbt_project.yml](/reference/model-configs#configuring-directories-of-models-in-dbt_projectyml)
- [Config inheritance](/reference/configs-and-properties#config-inheritance)
1 change: 1 addition & 0 deletions website/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -720,6 +720,7 @@ const sidebarSettings = {
"reference/resource-configs/limit",
"reference/resource-configs/severity",
"reference/resource-configs/store_failures",
"reference/resource-configs/store_failures_as",
"reference/resource-configs/where",
],
},
Expand Down

0 comments on commit 0eef471

Please sign in to comment.