Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

updated 1.7 migration guide to call out a behavior change #4271

Merged
merged 5 commits into from
Oct 12, 2023
Merged
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 17 additions & 5 deletions website/docs/guides/migration/versions/00-upgrading-to-v1.7.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,22 @@ description: New features and changes in dbt Core v1.7

dbt Labs is committed to providing backward compatibility for all versions 1.x, with the exception of any changes explicitly mentioned below. If you encounter an error upon upgrading, please let us know by [opening an issue](https://github.com/dbt-labs/dbt-core/issues/new).

### Behavior changes

dbt Core v1.7 expands the amount of sources you can configure freshness for. Previously, freshness was limited to sources with a `loaded_at_field`; now, freshness can be generated from warehouse metadata tables when available.

As part of this change, the `loaded_at_field` is no longer required to generate source freshness. If a source has a `freshness:` block, dbt will attempt to calculate freshness for that source:
- If a `loaded_at_field` is provided, dbt will calculate freshness via a select query (previous behavior).
- If a `loaded_at_field` is _not_ provided, dbt will calculate freshness via warehouse metadata tables when possible (new behavior).

This is technically a small behavior change comapared to previous versons. In order to _not_ calculate freshness for a source:
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved
- don't add a `freshness:` block
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved
- explicity set `freshness: null`
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved

## New and changed features and functionality

- [`dbt docs generate`](/reference/commands/cmd-docs) now supports `--select` to generate documentation for a subset of your project. Currently available for Snowflake and Postgres only, but other adapters are coming soon.
- [Source freshness](/docs/deploy/source-freshness) can now be generated from warehouse metadata tables, currently snowflake only, but other adapters that have metadata tables are coming soon. If you configure source freshness without a `loaded_at_field`, dbt will try to determine freshness from warehouse metadata tables.
- The nodes dictionary in the `catalog.json` can now be "partial" if `dbt docs generate` is run with a selector.
- [Source freshness](/docs/deploy/source-freshness) can now be generated from warehouse metadata tables, currently Snowflake only, but other adapters that have metadata tables are coming soon.

### MetricFlow enhancements

Expand All @@ -32,8 +43,9 @@ dbt Labs is committed to providing backward compatibility for all versions 1.x,

- The [manifest](/reference/artifacts/manifest-json) schema version has been updated to v11.
- The [run_results](/reference/artifacts/run-results-json) schema version has been updated to v5.
- Added [node attributes](/reference/artifacts/run-results-json) related to compilation (`compiled`, `compiled_code`, `relation_name`).

- There were a handful of specific changes to the [catalog.json](/reference/artifacts/catalog-json):
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved
- Added [node attributes](/reference/artifacts/run-results-json) related to compilation (`compiled`, `compiled_code`, `relation_name`) to the `catalog.json`.
- The nodes dictionary in the `catalog.json` can now be "partial" if `dbt docs generate` is run with a selector.

### Model governance

Expand All @@ -49,4 +61,4 @@ dbt Core v1.5 introduced model governance which we're continuing to refine. v1.
With these quick hits, you can now:
- Configure a `delimiter` for a seed file.
- Use packages with the same git repo and unique subdirectory.
- Moved the `date_spine` macro from dbt-utils to dbt-core.
- Access the `date_spine` macro directly from dbt-core (moved over from dbt-utils).
Loading