Skip to content

Commit

Permalink
New attributes in run_results.json (#4974)
Browse files Browse the repository at this point in the history
## What are you changing in this pull request and why?

Adding new attributes to `run_results.json`.

Closes #4125

## Checklist

Uncomment when publishing docs for a prerelease version of dbt:
- [x] Add versioning components, as described in [Versioning
Docs](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#versioning-entire-pages)
- [x] Add a note to the prerelease version [Migration
Guide](https://github.com/dbt-labs/docs.getdbt.com/tree/current/website/docs/docs/dbt-versions/core-upgrade)

- [x] Review the [Content style
guide](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/content-style-guide.md)
so my content adheres to these guidelines.
- [x] For [docs
versioning](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#about-versioning),
review how to [version a whole
page](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#adding-a-new-version)
and [version a block of
content](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#versioning-blocks-of-content).
  • Loading branch information
matthewshaver authored Feb 29, 2024
2 parents 70ddac5 + 9c5b945 commit fee085f
Show file tree
Hide file tree
Showing 2 changed files with 160 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,14 @@ Unit tests are defined in YML files in your `models/` directory and are currentl

The [`run`](/reference/commands/run#the-`--empty`-flag) and [`build`](/reference/commands/build#the---empty-flag) commands now support the `--empty` flag for building schema-only dry runs. The `--empty` flag limits the refs and sources to zero rows. dbt will still execute the model SQL against the target data warehouse but will avoid expensive reads of input data. This validates dependencies and ensures your models will build properly.

### Additional attributes in run_results.json

The run_results.json now includes three attributes related to the `applied` state that complement `unique_id`:

- `compiled`: Boolean entry of the node compilation status (`False` after parsing, but `True` after compiling).
- `compiled_code`: Rendered string of the code that was compiled (empty after parsing, but full string after compiling).
- `relation_name`: The fully-qualified name of the object that was (or will be) created/updated within the database.


## Quick hits

Expand Down
153 changes: 152 additions & 1 deletion website/docs/reference/artifacts/run-results-json.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,162 @@ Each entry in `results` is a [`Result` object](/reference/dbt-classes#result-obj
- `execution_time`: Total time spent executing this node
- `timing`: Array that breaks down execution time into steps (often `compile` + `execute`)
- `message`: How dbt will report this result on the CLI, based on information returned from the database

import RowsAffected from '/snippets/_run-result.md';

<RowsAffected/>

<!-- this partial comes from https://github.com/dbt-labs/docs.getdbt.com/tree/current/website/snippets/_run-result-->

<VersionBlock firstVersion="1.8">

The run_results.json includes three attributes related to the `applied` state that complement `unique_id`:

- `compiled`: Boolean entry of the node compilation status (`False` after parsing, but `True` after compiling).
- `compiled_code`: Rendered string of the code that was compiled (empty after parsing, but full string after compiling).
- `relation_name`: The fully-qualified name of the object that was (or will be) created/updated within the database.

Continue to look up additional information about the `logical` state of nodes using the full node object in manifest.json via the `unique_id`.

## Examples

Here are a few examples and the resulting output to the `run_results.json` file.

### Compile model results

Let's say that you have a model that looks like this:

<File name='models/my_model.sql'>

```sql
select {{ dbt.current_timestamp() }} as created_at
```

</File>

Compile the model:

```shell
dbt compile -s my_model
```

Here's a printed snippet from the `run_results.json`:

```json
{
"status": "success",
"timing": [
{
"name": "compile",
"started_at": "2023-10-12T16:35:28.510434Z",
"completed_at": "2023-10-12T16:35:28.519086Z"
},
{
"name": "execute",
"started_at": "2023-10-12T16:35:28.521633Z",
"completed_at": "2023-10-12T16:35:28.521641Z"
}
],
"thread_id": "Thread-2",
"execution_time": 0.0408780574798584,
"adapter_response": {},
"message": null,
"failures": null,
"unique_id": "model.my_project.my_model",
"compiled": true,
"compiled_code": "select now() as created_at",
"relation_name": "\"postgres\".\"dbt_dbeatty\".\"my_model\""
}
```

### Run generic data tests

Use the [`store_failures_as`](/reference/resource-configs/store_failures_as) config to store failures for only one data test in the database:

<File name='models/_models.yml'>

```yaml
models:
- name: my_model
columns:
- name: created_at
tests:
- not_null:
config:
store_failures_as: view
- unique:
config:
store_failures_as: ephemeral
```
</File>
Run the built-in `unique` test and store the failures as a table:

```shell
dbt test -s my_model
```

Here's a printed snippet from the `run_results.json`:

```json
"results": [
{
"status": "pass",
"timing": [
{
"name": "compile",
"started_at": "2023-10-12T17:20:51.279437Z",
"completed_at": "2023-10-12T17:20:51.317312Z"
},
{
"name": "execute",
"started_at": "2023-10-12T17:20:51.319812Z",
"completed_at": "2023-10-12T17:20:51.441967Z"
}
],
"thread_id": "Thread-2",
"execution_time": 0.1807551383972168,
"adapter_response": {
"_message": "SELECT 1",
"code": "SELECT",
"rows_affected": 1
},
"message": null,
"failures": 0,
"unique_id": "test.my_project.unique_my_model_created_at.a9276afbbb",
"compiled": true,
"compiled_code": "\n \n \n\nselect\n created_at as unique_field,\n count(*) as n_records\n\nfrom \"postgres\".\"dbt_dbeatty\".\"my_model\"\nwhere created_at is not null\ngroup by created_at\nhaving count(*) > 1\n\n\n",
"relation_name": null
},
{
"status": "pass",
"timing": [
{
"name": "compile",
"started_at": "2023-10-12T17:20:51.274049Z",
"completed_at": "2023-10-12T17:20:51.295237Z"
},
{
"name": "execute",
"started_at": "2023-10-12T17:20:51.296361Z",
"completed_at": "2023-10-12T17:20:51.491327Z"
}
],
"thread_id": "Thread-1",
"execution_time": 0.22345590591430664,
"adapter_response": {
"_message": "SELECT 1",
"code": "SELECT",
"rows_affected": 1
},
"message": null,
"failures": 0,
"unique_id": "test.my_project.not_null_my_model_created_at.9b412fbcc7",
"compiled": true,
"compiled_code": "\n \n \n\n\n\nselect *\nfrom \"postgres\".\"dbt_dbeatty\".\"my_model\"\nwhere created_at is null\n\n\n",
"relation_name": "\"postgres\".\"dbt_dbeatty_dbt_test__audit\".\"not_null_my_model_created_at\""
}
],
```

</VersionBlock>

0 comments on commit fee085f

Please sign in to comment.