Skip to content

dbt 0.21.0
Compare
Choose a tag to compare
@cmcarthur cmcarthur released this 04 Oct 18:44
· 21 commits to 0.21.latest since this release
v0.21.0
c009485

dbt Core 0.21.0 - Louis Kahn (October 4, 2021)

Breaking changes

  • Rename source freshness command, add full node selection, and align selection syntax with other tasks (#2987, #3554)
    • dbt source snapshot-freshness -> dbt source freshness (backwards compatible)
    • dbt source freshness --select source_name --> dbt source freshness --select source:source_name (not backwards compatible)
  • dbt-snowflake: Disable most transactional logic. See note below for details.

Features

  • Add dbt build command to run models, tests, seeds, and snapshots in DAG order. (#2743, #3490, #3608)
    • Skip downstream nodes during the build task when a test fails. (#3597, #3792)
    • Update cli and rpc flags for the build task to align with other commands (--resource-type, --store-failures) (#3596, #3884)
  • Introduce on_schema_change config to detect and handle schema changes on incremental models (#1132, #3387)
  • Capture changes to macros in state:modified. Introduce new state: sub-selectors: modified.body, modified.configs, modified.persisted_descriptions, modified.relation, modified.macros (#2704, #3278, #3559)
  • Enable setting configs in schema files for models, seeds, snapshots, analyses, tests (#2401, #3616)
  • Make --models and --select synonyms, except for ls (to preserve existing behavior) (#3210, #3791)
  • Customize ls task JSON output by adding new flag --output-keys (#3778, #3395)
  • Added default field in the selectors.yml to allow user to define default selector (#3448, #3875, #3892)
  • Log tests that are not indirectly selected. Add --greedy flag to test, list, build and greedy property in yaml selectors (#3723, #3833)
  • Scrub environment secrets prefixed with DBT_ENV_SECRET_ from logs and console output (#3440, #3617)

Artifacts

  • Added timing and thread information to sources.json artifact (#3804, #3894)
  • Bump artifact schema versions for 0.21.0 (#3945)

Fixes

  • Fix type coercion issues when fetching query result sets (#2984, #3499)
  • Handle whitespace after a plus sign on the project config (#3526)
  • Fix table and view materialization issue when switching from one to the other (#2161), #3547)
  • Avoid caching schemas for tests when store_failures is not enabled (#3715, #3716)
  • dbt debug shows a summary of whether all checks passed or not (#3831, #3832)
  • Fix bug with initializing a dataclass that inherits from typing.Protocol, specifically for dbt.config.profile.Profile (#3843, #3855)
  • Introduce a macro, get_where_subquery, for tests that use where config. Alias filtering subquery as dbt_subquery instead of resource identifier (#3857, #3859)
  • Use group by column_name in accepted_values test for compatibility with most database engines (#3905, #3906)
  • Avoid infinite recursion in state:modified.macros check (#3904, #3957)
  • Cast log messages to strings before scrubbing of prefixed env vars (#3971, #3972)

Under the hood

  • Improve default view and table materialization performance by checking relational cache before attempting to drop temp relations (#3112, #3468)
  • Move the example project used by dbt init into dbt repository, to avoid cloning an external repo (#3005, #3474, #3536)
  • Better interaction between dbt init and adapters. Avoid raising errors while initializing a project (#2814, #3483)
  • Update create_adapter_plugins script to include latest accessories, and stay up to date with latest dbt-core version (#3002, #3509)
  • Allow the default seed macro's SQL parameter, %s, to be replaced by dispatching a new macro, get_binding_char(). This enables adapters with parameter marker characters such as ? to not have to override basic_load_csv_rows. (#3622, #3623)
  • Add adapter_unique_id to invocation context in anonymous usage tracking, to better understand dbt adoption (#3713, #3796)
  • Specify macro_namespace = 'dbt' for all dispatched macros in the global project, making it possible to dispatch to macro implementations defined in packages. Dispatch generate_schema_name and generate_alias_name (#3456, #3851)

Packages

  • Better dbt hub registry packages version logging that prompts the user for upgrades to relevant packages (#3560, #3763, #3759)
  • Alert users on package rename (hub.getdbt.com#180, #3825)
  • Retry transient GitHub failures during download (#3546, #3729)

Performance

  • Experimental parser now detects macro overrides of ref, source, and config builtins. (#3581, #3582)
  • Add performance regression testing #3602
  • Don't reload and validate schema files if they haven't changed (#3563, #3888)

Dependencies

  • Require werkzeug>=1 (#3590)

Plugins

dbt-postgres

  • Add optional sslcert, sslkey, and sslrootcert profile arguments to the Postgres connector. (#3472, #3473)
  • Add connect_timeout profile configuration for Postgres and child adapters. (#3581, #3582)

dbt-redshift

  • Introduce ra3 profile configuration on Redshift to support cross-db sources (#3236, #3408)

dbt-snowflake

  • Breaking: Turn off transactions and turn on autocommit by default. Within dbt materializations, wrap DML statements in explicit begin and commit. Note that it is not recommended to run DML statements outside of dbt materialization logic. If you do this, despite our recommendation, you will need to wrap those statements in explicit begin and commit. Note also that this may affect user-space code that depends on transactions, such as pre-hooks and post-hooks that specify transaction: true or transaction: false. We recommend removing those references to transactions. (#2748, #3480, #3510)
  • Fix batching for large seeds on Snowflake (#3941, #3942)

dbt-bigquery

  • Enhance BigQuery copy materialization (#3570, #3606):
    • to simplify config (default usage of copy_materialization='table' if is is not found in global or local config)
    • to let copy several source tables into single target table at a time. (Google doc reference)
  • Add support for execution_project on BigQuery through profile configuration (#3707, #3708)
  • Support BigQuery-specific aliases target_dataset and target_project in snapshot configs (#3694, #3834)
  • Separated table vs view configuration for BigQuery since some configuration is not possible to set for tables vs views. (#3682, #3691)

dbt-rpc

  • Fix for RPC requests that raise a RecursionError when serializing Undefined values as JSON (#3464, #3687)
  • Add build RPC method, and a subset of flags for build task (#3595, #3674)

Contributors

Thank you all for your contributions to dbt! 🎉

Louis Kahn (1901–1974)

Thanks to @grantwinship for writing this biography of Louis Kahn—a famous Philadelphian, and the namesake of this dbt Core release.

The architecture of Louis Kahn inimitably and improbably unites the forces of mass and light. His often hefty concrete structures embrace weighty, intentional interior spaces, huge shapes and curves, and angled baths of light to create a take on mid-century brutalism that is suffused with a utopian optimism and playfulness.

Born in the Russian Empire, in what is now Estonia, Itze-Leib Schmuilowsky emigrated to Philadelphia in 1906 at age four. He became a US citizen in 1914, and changed his name, along with his family, to Louis Kahn in 1915. A talented artist from a young age, he repeatedly won Philadelphia's annual award for best watercolor by a high school student, but turned down a full scholarship to the Pennsylvania Academy of the Fine Arts to pursue architecture at the University of Pennsylvania.

There he studied a restrained variant of the Beaux-Arts school with Paul Philippe Cret, who influenced the young Kahn greatly in his dedication to grand form and proportion, if not the style's signature ornamentation. Kahn would, after achieving international renown, return to Philadelphia to teach at the University of Pennsylvania from 1957 until his death, becoming the Paul Philippe Cret Professor of Architecture.

First though, he cut his teeth taking public housing projects in Philadelphia, many of which remain unbuilt, but are revered by students of architecture. It was a trip abroad through the Mediterranean in 1950, seeing the ruins and monuments of Egypt and Greece again as an established architect, that drove him to push his personal style further. He returned and built an extension to Yale University's Art Gallery that immediately established him as unique voice in American architecture. He would over the next decade and a half design many other works of genius, chief among them the Salk Institute in La Jolla, California and what many consider to be his masterwork, the Jatiya Sangsad Bhaban government building that serves as the capitol of Bangladesh. Both buildings' balance of massive concrete geometries with the light, space, and environment of their locations create a breathtaking aesthetic that is simultaneously ancient and futuristic. Experiencing either of them, the immense impact that Kahn has had on modern architecture is immediately apparent.

Assets 2
Loading