Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New attributes in run_results.json #4974

Merged
merged 9 commits into from
Feb 29, 2024
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,14 @@ Unit tests are defined in YML files in your `models/` directory and are currentl

The [`run`](/reference/commands/run#the-`--empty`-flag) and [`build`](/reference/commands/build#the---empty-flag) commands now support the `--empty` flag for building schema-only dry runs. The `--empty` flag limits the refs and sources to zero rows. dbt will still execute the model SQL against the target data warehouse but will avoid expensive reads of input data. This validates dependencies and ensures your models will build properly.

### Additional attributes in run_results.json

The run_results.json now includes three attributes related to the `applied` state that complement `unique_id`:

- `compiled`: Boolean entry of the node compilation status (`False` after parsing, but `True` after compiling).
- `compiled_code`: Rendered string of the code that was compiled (empty after parsing, but full string after compiling).
- `relation_name`: The fully-qualified name of the object that was (or will be) created/updated within the database.


## Quick hits

Expand Down
153 changes: 152 additions & 1 deletion website/docs/reference/artifacts/run-results-json.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,162 @@ Each entry in `results` is a [`Result` object](/reference/dbt-classes#result-obj
- `execution_time`: Total time spent executing this node
- `timing`: Array that breaks down execution time into steps (often `compile` + `execute`)
- `message`: How dbt will report this result on the CLI, based on information returned from the database

import RowsAffected from '/snippets/_run-result.md';

<RowsAffected/>

<!-- this partial comes from https://github.com/dbt-labs/docs.getdbt.com/tree/current/website/snippets/_run-result-->

<VersionBlock firstVersion="1.8">

The run_results.json includes three attributes related to the `applied` state that complement `unique_id`:

- `compiled`: Boolean entry of the node compilation status (`False` after parsing, but `True` after compiling).
- `compiled_code`: Rendered string of the code that was compiled (empty after parsing, but full string after compiling).
- `relation_name`: The fully-qualified name of the object that was (or will be) created/updated within the database.

Continue to look up additional information about the `logical` state of nodes using the full node object in manifest.json via the `unique_id`.

## Examples

Here are a few examples and the resulting output to the `run_results.json` file.

### Compile model results

Let's say that you have a model that looks like this:

<File name='models/my_model.sql'>

```sql
select {{ dbt.current_timestamp() }} as created_at
```

</File>

Compile the model:

```shell
dbt compile -s my_model
```

Here's a printed snippet from the `run_results.json`:

```json
{
"status": "success",
"timing": [
{
"name": "compile",
"started_at": "2023-10-12T16:35:28.510434Z",
"completed_at": "2023-10-12T16:35:28.519086Z"
},
{
"name": "execute",
"started_at": "2023-10-12T16:35:28.521633Z",
"completed_at": "2023-10-12T16:35:28.521641Z"
}
],
"thread_id": "Thread-2",
"execution_time": 0.0408780574798584,
"adapter_response": {},
"message": null,
"failures": null,
"unique_id": "model.my_project.my_model",
"compiled": true,
"compiled_code": "select now() as created_at",
"relation_name": "\"postgres\".\"dbt_dbeatty\".\"my_model\""
}
```

### Run generic data tests

Use the [`store_failures_as`](/reference/resource-configs/store_failures_as) config to store failures for only one data test in the database:

<File name='models/_models.yml'>

```yaml
models:
- name: my_model
columns:
- name: created_at
tests:
- not_null:
config:
store_failures_as: view
- unique:
config:
store_failures_as: ephemeral
```

</File>

Run the built-in `unique` test and store the failures as a table:

```shell
dbt test -s my_model
```

Here's a printed snippet from the `run_results.json`:

```json
"results": [
{
"status": "pass",
"timing": [
{
"name": "compile",
"started_at": "2023-10-12T17:20:51.279437Z",
"completed_at": "2023-10-12T17:20:51.317312Z"
},
{
"name": "execute",
"started_at": "2023-10-12T17:20:51.319812Z",
"completed_at": "2023-10-12T17:20:51.441967Z"
}
],
"thread_id": "Thread-2",
"execution_time": 0.1807551383972168,
"adapter_response": {
"_message": "SELECT 1",
"code": "SELECT",
"rows_affected": 1
},
"message": null,
"failures": 0,
"unique_id": "test.my_project.unique_my_model_created_at.a9276afbbb",
"compiled": true,
"compiled_code": "\n \n \n\nselect\n created_at as unique_field,\n count(*) as n_records\n\nfrom \"postgres\".\"dbt_dbeatty\".\"my_model\"\nwhere created_at is not null\ngroup by created_at\nhaving count(*) > 1\n\n\n",
"relation_name": null
},
{
"status": "pass",
"timing": [
{
"name": "compile",
"started_at": "2023-10-12T17:20:51.274049Z",
"completed_at": "2023-10-12T17:20:51.295237Z"
},
{
"name": "execute",
"started_at": "2023-10-12T17:20:51.296361Z",
"completed_at": "2023-10-12T17:20:51.491327Z"
}
],
"thread_id": "Thread-1",
"execution_time": 0.22345590591430664,
"adapter_response": {
"_message": "SELECT 1",
"code": "SELECT",
"rows_affected": 1
},
"message": null,
"failures": 0,
"unique_id": "test.my_project.not_null_my_model_created_at.9b412fbcc7",
"compiled": true,
"compiled_code": "\n \n \n\n\n\nselect *\nfrom \"postgres\".\"dbt_dbeatty\".\"my_model\"\nwhere created_at is null\n\n\n",
"relation_name": "\"postgres\".\"dbt_dbeatty_dbt_test__audit\".\"not_null_my_model_created_at\""
}
],
```

matthewshaver marked this conversation as resolved.
Show resolved Hide resolved
</VersionBlock>
Loading