Skip to content

Commit

Permalink
add artifacts_upload_packages var to get_dataset_content
Browse files Browse the repository at this point in the history
  • Loading branch information
npeshkov committed Dec 9, 2024
1 parent 91d0a6e commit f188380
Show file tree
Hide file tree
Showing 2 changed files with 15 additions and 0 deletions.
11 changes: 11 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,17 @@ Note that model materializations and `on_schema_change` configs are defined in t

> Configurations made in your dbt_project.yml file will override any configurations in a package (either in the dbt_project.yml file of the package, or in config blocks).

### Selection of packages to upload

By default, all packages are uploaded. Some users may be interested in choosing which packages to upload from non-execution datasets. Thus, we allow setting `artifacts_upload_packages` variable:

```yml
vars:
artifacts_upload_packages: ['my_package']
```

> Note: execution-type artifacts are not affected by this filter.

### Environment Variables

If the project is running in dbt Cloud, the following five columns (<https://docs.getdbt.com/docs/dbt-cloud/using-dbt-cloud/cloud-environment-variables#special-environment-variables>) will be automatically populated in the fct_dbt__invocations model:
Expand Down
4 changes: 4 additions & 0 deletions macros/upload_results/get_dataset_content.sql
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@
{% elif dataset in ['seeds', 'snapshots', 'tests', 'models'] %}
{# Use the nodes in the [graph](https://docs.getdbt.com/reference/dbt-jinja-functions/graph) to extract details #}
{% set objects = graph.nodes.values() | selectattr("resource_type", "equalto", dataset[:-1]) | list %}
{# Optionally, filter by package_name #}
{% if var("artifacts_upload_packages", "all") != "all" %}
{% set objects = (objects | selectattr("package_name", "in", var("artifacts_upload_packages")) | list) %}
{% endif %}
{% elif dataset in ['exposures', 'sources'] %}
{# Use the [graph](https://docs.getdbt.com/reference/dbt-jinja-functions/graph) to extract details #}
{% set objects = graph.get(dataset).values() | list %}
Expand Down

0 comments on commit f188380

Please sign in to comment.