Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New attributes in run_results.json #4974

Merged
merged 9 commits into from
Feb 29, 2024
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,14 @@ Unit tests are defined in YML files in your `models/` directory and are currentl

The [`run`](/reference/commands/run#the-`--empty`-flag) and [`build`](/reference/commands/build#the---empty-flag) commands now support the `--empty` flag for building schema-only dry runs. The `--empty` flag limits the refs and sources to zero rows. dbt will still execute the model SQL against the target data warehouse but will avoid expensive reads of input data. This validates dependencies and ensures your models will build properly.

### Additional attributes in run_results.json

The run_results.json now includes three attributes related to the `applied` state that complement `unique_id`:

- `compiled`: Boolean entry of the node compilation status (`False` after parsing, but `True` after compiling).
- `compiled_code`: Rendered string of the code that was compiled (empty after parsing, but full string after compiling).
- `relation_name`: The fully-qualified name of the object that was (or will be) created/updated within the database.


## Quick hits

Expand Down
153 changes: 152 additions & 1 deletion website/docs/reference/artifacts/run-results-json.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,162 @@ Each entry in `results` is a [`Result` object](/reference/dbt-classes#result-obj
- `execution_time`: Total time spent executing this node
- `timing`: Array that breaks down execution time into steps (often `compile` + `execute`)
- `message`: How dbt will report this result on the CLI, based on information returned from the database

import RowsAffected from '/snippets/_run-result.md';

<RowsAffected/>

<!-- this partial comes from https://github.com/dbt-labs/docs.getdbt.com/tree/current/website/snippets/_run-result-->

<VersionBlock firstVersion="1.8">

The run_results.json includes three attributes related to the `applied` state that complement `unique_id`:

- `compiled`: Boolean entry of the node compilation status (`False` after parsing, but `True` after compiling).
- `compiled_code`: Rendered string of the code that was compiled (empty after parsing, but full string after compiling).
- `relation_name`: The fully-qualified name of the object that was (or will be) created/updated within the database.

Any additional information about the `logical` state of nodes should continue to be looked up in the full node object in manifest.json via the unique_id.
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved

matthewshaver marked this conversation as resolved.
Show resolved Hide resolved
## Examples

Here are a few examples and the resulting output to the `run_results.json` file.

### Compile model results

Let's say that you have a model that looks like this:

<File name='models/my_model.sql'>

```sql
select {{ dbt.current_timestamp() }} as created_at
```

</File>

Compile the model:

```shell
dbt compile -s my_model
```

This is a printed snippet from the `run_results.json`:
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved

```json
{
"status": "success",
"timing": [
{
"name": "compile",
"started_at": "2023-10-12T16:35:28.510434Z",
"completed_at": "2023-10-12T16:35:28.519086Z"
},
{
"name": "execute",
"started_at": "2023-10-12T16:35:28.521633Z",
"completed_at": "2023-10-12T16:35:28.521641Z"
}
],
"thread_id": "Thread-2",
"execution_time": 0.0408780574798584,
"adapter_response": {},
"message": null,
"failures": null,
"unique_id": "model.my_project.my_model",
"compiled": true,
"compiled_code": "select now() as created_at",
"relation_name": "\"postgres\".\"dbt_dbeatty\".\"my_model\""
}
```

### Run generic data tests

Use the [`store_failures_as`](/reference/resource-configs/store_failures_as) config to store failures for only one data test in the database:

<File name='models/_models.yml'>

```yaml
models:
- name: my_model
columns:
- name: created_at
tests:
- not_null:
config:
store_failures_as: view
- unique:
config:
store_failures_as: ephemeral
```

</File>

Run the built-in `unique` test and store the failures as a table:

```shell
dbt test -s my_model
```

This is a printed snippet from the `run_results.json`:
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved

```json
"results": [
{
"status": "pass",
"timing": [
{
"name": "compile",
"started_at": "2023-10-12T17:20:51.279437Z",
"completed_at": "2023-10-12T17:20:51.317312Z"
},
{
"name": "execute",
"started_at": "2023-10-12T17:20:51.319812Z",
"completed_at": "2023-10-12T17:20:51.441967Z"
}
],
"thread_id": "Thread-2",
"execution_time": 0.1807551383972168,
"adapter_response": {
"_message": "SELECT 1",
"code": "SELECT",
"rows_affected": 1
},
"message": null,
"failures": 0,
"unique_id": "test.my_project.unique_my_model_created_at.a9276afbbb",
"compiled": true,
"compiled_code": "\n \n \n\nselect\n created_at as unique_field,\n count(*) as n_records\n\nfrom \"postgres\".\"dbt_dbeatty\".\"my_model\"\nwhere created_at is not null\ngroup by created_at\nhaving count(*) > 1\n\n\n",
"relation_name": null
},
{
"status": "pass",
"timing": [
{
"name": "compile",
"started_at": "2023-10-12T17:20:51.274049Z",
"completed_at": "2023-10-12T17:20:51.295237Z"
},
{
"name": "execute",
"started_at": "2023-10-12T17:20:51.296361Z",
"completed_at": "2023-10-12T17:20:51.491327Z"
}
],
"thread_id": "Thread-1",
"execution_time": 0.22345590591430664,
"adapter_response": {
"_message": "SELECT 1",
"code": "SELECT",
"rows_affected": 1
},
"message": null,
"failures": 0,
"unique_id": "test.my_project.not_null_my_model_created_at.9b412fbcc7",
"compiled": true,
"compiled_code": "\n \n \n\n\n\nselect *\nfrom \"postgres\".\"dbt_dbeatty\".\"my_model\"\nwhere created_at is null\n\n\n",
"relation_name": "\"postgres\".\"dbt_dbeatty_dbt_test__audit\".\"not_null_my_model_created_at\""
}
],
```

</VersionBlock>
Loading