dbt 0.19.1b2

A second beta release of v0.19.1.

dbt 0.19.0

dbt 0.19.0 - Kiyoshi Kuromiya (January 27, 2021)

Links:

• 💬 Discussion: v0.19.0
• 📖 New and changed documentation

Breaking changes

• The format for sources.json, run-results.json, manifest.json, and catalog.json has changed:
  • Each now has a common metadata dictionary (#2761, #2778). The contents include: schema and dbt versions (#2670, #2767); invocation_id (#2763, #2784); custom environment variables prefixed with DBT_ENV_CUSTOM_ENV_ (#2764, #2785); cli and rpc arguments in the run_results.json (#2510, #2813).
  • Remove injected_sql from manifest nodes, use compiled_sql instead (#2762, #2834)
• Defer if and only if upstream reference does not exist in current environment namespace (#2909, #2946)
• Rationalize run result status reporting and clean up artifact schema (#2493, #2943)
• Add adapter specific query execution info to run results and source freshness results artifacts. Statement call blocks return response instead of status, and the adapter method get_status is now get_response (#2747, #2961)

Deprecations

• Removed support for config-version: 1 of dbt_project.yml, which was deprecated in v0.17.0. Use config-version: 2 in all projects and installed packages.

Features

Core

• dbt will compare configurations using the un-rendered form of the config block in dbt_project.yml. (#2713, #2735)
• Added state and defer arguments to the RPC client, matching the CLI (#2678, #2736)
• Added ability to snapshot hard-deleted records (opt-in with invalidate_hard_deletes config option). (#249, #2749)
• Added revival for snapshotting hard-deleted records. (#2819, #2821)
• Improved error messages for YAML selectors (#2700, #2781)
• Added --defer flag for dbt test as well (#2701, #2954)
• Added native python re module for regex in jinja templates (#1755, #1755)
• Store resolved node names in manifest (#2647, #2837)
• Save selectors dictionary to manifest, allow descriptions (#2693, #2866)
• Normalize cli-style-strings in manifest selectors dictionary (#2879, #2895)

Docs website

• Add select/deselect option in DAG view dropups. (docs#98, docs#138)
• Fixed issue where sources with tags were not showing up in graph viz (docs#93, docs#139)
• Use compiled_sql instead of injected_sql for "Compiled" (docs#146, docs#148)
• Fixed issue where data tests with tags were not showing up in graph viz (docs#147, docs#157)

BigQuery

• Added dbt_invocation_id for each BigQuery job to enable performance analysis (#2808, #2809)
• Added support for BigQuery connections using refresh tokens (#2344, #2805)
• Added macro get_partitions_metadata(table) to return partition metadata for BigQuery partitioned tables (#2552, #2596)
• Hourly, monthly and yearly partitions available in BigQuery (#2476, #2903)
• Allow BigQuery to default to the environment's default project (#2828, #2908)

Fixes

• Respect --project-dir in dbt clean command (#2840, #2841)
• Fix Redshift adapter get_columns_in_relation macro to push schema filter down to the svv_external_columns view (#2854, #2854)
• Increased the supported relation name length in postgres from 29 to 51 (#2850)
• dbt list command always return 0 as exit code (#2886, #2892)
• Set default materialized for test node configs to test (#2806, #2902)
• Allow docs blocks in exposure descriptions (#2913, #2920)
• Use original file path instead of absolute path as checksum for big seeds (#2927, #2939)
• Fix KeyError if deferring to a manifest with a since-deleted source, ephemeral model, or test (#2875, #2958)
• Fix regression with defining exposures and other resources with the same name (#2969, #3009)
• Remove ellipses printed while parsing (#2971, #2996)

Under the hood

• Save manifest.json at the same time we save the run_results.json at the end of a run (#2765, #2799)
• Added strategy-specific validation to improve the relevancy of compilation errors for the timestamp and check snapshot strategies. ((#2787, #2791)
• Changed rpc test timeouts to avoid locally run test failures (#2803,#2804)
• Added a debug_query on the base adapter that will allow plugin authors to create custom debug queries (#2751,#2871)
• Add unixodbc-dev package to testing docker image (#2859)
• Add event tracking for project parser/load times (#2823,#2893)
• Bump cryptography version to >= 3.2 and bump snowflake connector to 2.3.6 (#2896, #2922)
• Widen supported Google Cloud libraries dependencies (#2794, #2877).
• Bump hologram version to 0.0.11. Add scripts/dtr.py (#2888,#2889)
• Bump hologram version to 0.0.12. Ad...

dbt 0.19.0rc2

An initial release of 0.19.0rc2

dbt 0.19.0rc1

An initial release candidate of v0.19.0

dbt 0.19.0b1

An initial release of 0.19.0b1

dbt 0.18.1

Features

• Specify all three logging levels (INFO, WARNING, ERROR) in result logs for commands test, seed, run, snapshot and source snapshot-freshness (#2680, #2723)
• Added "exposures" (#2730, #2752, #2777)
• Added 'Last Modified' stat in snowflake catalog macro. Now should be available in docs. (#2728)
• Added retry support for rateLimitExceeded error from BigQuery, (#2795, #2796)

Fixes

• dbt compile and dbt run failed with KeyError: 'endpoint_resolver' when threads > 1 and method: iam had been specified in the profiles.yaml (#2756, #2766)
• Fix Redshift adapter to include columns from external tables when using the get_columns_in_relation macro (#2753, #2754)

Docs

• Add Exposure nodes (docs#135, docs#136, docs#137)

Under the hood

• If column config says quote, use quoting in SQL for adding a comment. (#2539, #2733)
• Added support for running docker-based tests under Linux. (#2739)
• Require extra snowflake-connector-python[secure-local-storage] on all dbt-snowflake installations (#2779, #2789)

Contributors:

• @tpilewicz (#2723)
• @heisencoder (#2739)
• @Mr-Nobody99 (#2732)
• @jweibel22 (#2766)
• @aiguofer (#2754)
• @championj-foxtel (#2796)

dbt 0.18.1rc1

An initial release of 0.18.1rc1

dbt 0.18.1b3

An initial release of 0.18.1b3

dbt 0.18.1b2

An initial release of 0.18.1b2

v0.18.1b1

Merge remote-tracking branch 'origin/releases/0.18.1b1' into dev/0.18.1