From 5b1dd5f9fb7378327b672bafb5b87cc70d0caf41 Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Mon, 9 Oct 2023 16:47:20 -0400 Subject: [PATCH 1/8] v1.7 Migration Guide updates --- .../versions/00-upgrading-to-v1.7.md | 35 ++++++++++++++++++- 1 file changed, 34 insertions(+), 1 deletion(-) diff --git a/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md b/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md index 036c734dfb1..56f08cc3696 100644 --- a/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md +++ b/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md @@ -18,7 +18,40 @@ dbt Labs is committed to providing backward compatibility for all versions 1.x, **COMING SOON** +## New and changed documentation + +- `dbt docs generate` now supports `--select` to generate documentation for a subset of your project +- currently Snowflake and Postgres only, but other adapters on their way + +### source freshness can now be generated from warehouse metadata tables +- currently snowflake only, but other adapters that have metadata tables on their way +- if you configure source freshness without a `loaded_at_field`, dbt will try to determine freshness from warehouse metadata tables + +### MetricFlow enhancements +- automatically create metrics on measures, `create_metric: true` +- optional `label` in semantic_models, measures, dimensions and + entities +- new configurations for semantic models - enable/disable, group, meta +- support `fill_nulls_with` and `join_to_timespine` for metric nodes +- `saved_queries` extend governance beyond the semantic objects to their consumption + +## For consumers of dbt artifacts (metadata) + +The [manifest](https://docs.getdbt.com/reference/artifacts/manifest-json) schema version has been updated to v11. Specific changes: +- ??? + +@dbeatty10 do you know what the manifest schema changes were? The [PR that bumped the version](https://github.com/dbt-labs/dbt-core/pull/8335) didn't seem to make any other changes. + +The [run_results](https://docs.getdbt.com/reference/artifacts/run-results-json) schema version has been updated to v5. Specific changes: +- Add node attributes related to compilation (`compiled`, `compiled_code`, `relation_name`) + +The nodes dictionary in the `catalog.json` can now be "partial" if `dbt docs generate` is run with a selector. + ### Quick hits -**COMING SOON** +- You can configure a `delimiter` for a seed file. +- Allow setting `access` as a config. +- Support packages with same git repo and unique subdirectory. +- Moved `date_spine` macro from dbt-utils to dbt-core. + From f6ffe33b3d670b4fa2556925b34e887338ae5fc5 Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Wed, 11 Oct 2023 15:45:16 -0400 Subject: [PATCH 2/8] Updates to migration guide --- .../versions/00-upgrading-to-v1.7.md | 50 ++++++++----------- 1 file changed, 22 insertions(+), 28 deletions(-) diff --git a/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md b/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md index 56f08cc3696..64a4a4d2259 100644 --- a/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md +++ b/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md @@ -14,44 +14,38 @@ description: New features and changes in dbt Core v1.7 dbt Labs is committed to providing backward compatibility for all versions 1.x, with the exception of any changes explicitly mentioned below. If you encounter an error upon upgrading, please let us know by [opening an issue](https://github.com/dbt-labs/dbt-core/issues/new). -### Behavior changes +## New and changed features and functionality -**COMING SOON** +- [`dbt docs generate`](/reference/commands/cmd-docs) now supports `--select` to generate documentation for a subset of your project. Currently available for Snowflake and Postgres only, but other adapters are coming soon. +- [Source freshness](/docs/deploy/source-freshness) can now be generated from warehouse metadata tables, currently snowflake only, but other adapters that have metadata tables are coming soon. If you configure source freshness without a `loaded_at_field`, dbt will try to determine freshness from warehouse metadata tables. +- The nodes dictionary in the `catalog.json` can now be "partial" if `dbt docs generate` is run with a selector. -## New and changed documentation - -- `dbt docs generate` now supports `--select` to generate documentation for a subset of your project -- currently Snowflake and Postgres only, but other adapters on their way +### MetricFlow enhancements -### source freshness can now be generated from warehouse metadata tables -- currently snowflake only, but other adapters that have metadata tables on their way -- if you configure source freshness without a `loaded_at_field`, dbt will try to determine freshness from warehouse metadata tables +- Automatically create metrics on measures with `create_metric: true`. +- Optional [`label`](/docs/build/semantic-models) in semantic_models, measures, dimensions and entities. +- New configurations for semantic models - [enable/disable](/reference/resource-configs/enabled), [group](/reference/resource-configs/group_), and [meta](/reference/resource-configs/meta). +- Support `fill_nulls_with` and `join_to_timespine` for metric nodes. +- `saved_queries` extends governance beyond the semantic objects to their consumption. -### MetricFlow enhancements -- automatically create metrics on measures, `create_metric: true` -- optional `label` in semantic_models, measures, dimensions and - entities -- new configurations for semantic models - enable/disable, group, meta -- support `fill_nulls_with` and `join_to_timespine` for metric nodes -- `saved_queries` extend governance beyond the semantic objects to their consumption +### For consumers of dbt artifacts (metadata) -## For consumers of dbt artifacts (metadata) +- The [manifest](/reference/artifacts/manifest-json) schema version has been updated to v11. +- The [run_results](/reference/artifacts/run-results-json) schema version has been updated to v5. +- Added [node attributes](/reference/artifacts/run-results-json) related to compilation (`compiled`, `compiled_code`, `relation_name`). -The [manifest](https://docs.getdbt.com/reference/artifacts/manifest-json) schema version has been updated to v11. Specific changes: -- ??? -@dbeatty10 do you know what the manifest schema changes were? The [PR that bumped the version](https://github.com/dbt-labs/dbt-core/pull/8335) didn't seem to make any other changes. +### Model governance -The [run_results](https://docs.getdbt.com/reference/artifacts/run-results-json) schema version has been updated to v5. Specific changes: -- Add node attributes related to compilation (`compiled`, `compiled_code`, `relation_name`) +Model governance was introduced in dbt Core v1.5 and it is continuing to be refined. The additional features and functionality have been added in v1.7 are as follows: -The nodes dictionary in the `catalog.json` can now be "partial" if `dbt docs generate` is run with a selector. +- **[Breaking change detection](/reference/resource-properties/versions) for models with contracts enforced:** When dbt detects a breaking change to a model with an enforced contract during state comparison, it will now raise an error for versioned models and a warning for models that are not versioned. +- **[Set `access` as a config](/reference/resource-configs/access):** It's now possible to set a model's `access` within config blocks in the model's file or the `dbt_project.yml` for an entire subfolder at once. +- **[Type aliasing for model contracts](/reference/resource-configs/contract):** dbt will use each adapter's built-in type aliasing for user-provided data types—meaning you can now write `string` always, and dbt will translate to `text` on Postgres/Redshift. This is on by default, and it's possible to opt-out. +- **[Raise warning for numeric types](/reference/resource-configs/contract):** This led to issues when putting `numeric` in model contracts without considering that Snowflake interprets it as `numeric(38,0)`, and will round decimals accordingly. dbt will raise a warning if it sees a numeric type without specified precision/scale. ### Quick hits - You can configure a `delimiter` for a seed file. -- Allow setting `access` as a config. -- Support packages with same git repo and unique subdirectory. -- Moved `date_spine` macro from dbt-utils to dbt-core. - - +- Support packages with the same git repo and unique subdirectory. +- Moved the `date_spine` macro from dbt-utils to dbt-core. \ No newline at end of file From 853f18764b8117d32b745e85e6c389619de2086f Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Wed, 11 Oct 2023 15:55:15 -0400 Subject: [PATCH 3/8] Update website/docs/guides/migration/versions/00-upgrading-to-v1.7.md --- website/docs/guides/migration/versions/00-upgrading-to-v1.7.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md b/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md index 64a4a4d2259..4ad87a40557 100644 --- a/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md +++ b/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md @@ -24,7 +24,7 @@ dbt Labs is committed to providing backward compatibility for all versions 1.x, - Automatically create metrics on measures with `create_metric: true`. - Optional [`label`](/docs/build/semantic-models) in semantic_models, measures, dimensions and entities. -- New configurations for semantic models - [enable/disable](/reference/resource-configs/enabled), [group](/reference/resource-configs/group_), and [meta](/reference/resource-configs/meta). +- New configurations for semantic models - [enable/disable](/reference/resource-configs/enabled), [group](/reference/resource-configs/group), and [meta](/reference/resource-configs/meta). - Support `fill_nulls_with` and `join_to_timespine` for metric nodes. - `saved_queries` extends governance beyond the semantic objects to their consumption. From 7d2c8587323b364598d651b9ddeff3e8d055ba43 Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Wed, 11 Oct 2023 18:42:50 -0400 Subject: [PATCH 4/8] Update 00-upgrading-to-v1.7.md --- .../docs/guides/migration/versions/00-upgrading-to-v1.7.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md b/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md index 4ad87a40557..4ffb2220006 100644 --- a/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md +++ b/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md @@ -8,7 +8,7 @@ description: New features and changes in dbt Core v1.7 - [Changelog](https://github.com/dbt-labs/dbt-core/blob/8aaed0e29f9560bc53d9d3e88325a9597318e375/CHANGELOG.md) - [CLI Installation guide](/docs/core/installation) - [Cloud upgrade guide](/docs/dbt-versions/upgrade-core-in-cloud) -- [Release schedule](https://github.com/dbt-labs/dbt-core/issues/7481) +- [Release schedule](https://github.com/dbt-labs/dbt-core/issues/8260) ## What to know before upgrading @@ -48,4 +48,4 @@ Model governance was introduced in dbt Core v1.5 and it is continuing to be refi - You can configure a `delimiter` for a seed file. - Support packages with the same git repo and unique subdirectory. -- Moved the `date_spine` macro from dbt-utils to dbt-core. \ No newline at end of file +- Moved the `date_spine` macro from dbt-utils to dbt-core. From e4f2460b77e18ab4b92c2901549600ccd9b94ebd Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Wed, 11 Oct 2023 18:44:05 -0400 Subject: [PATCH 5/8] Update website/docs/guides/migration/versions/00-upgrading-to-v1.7.md Co-authored-by: Leona B. Campbell <3880403+runleonarun@users.noreply.github.com> --- website/docs/guides/migration/versions/00-upgrading-to-v1.7.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md b/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md index 4ffb2220006..fe2eccce972 100644 --- a/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md +++ b/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md @@ -37,7 +37,7 @@ dbt Labs is committed to providing backward compatibility for all versions 1.x, ### Model governance -Model governance was introduced in dbt Core v1.5 and it is continuing to be refined. The additional features and functionality have been added in v1.7 are as follows: +dbt Core v1.5 introduced model governance which we're continuing to refine. v1.7 includes these additional features and functionality: - **[Breaking change detection](/reference/resource-properties/versions) for models with contracts enforced:** When dbt detects a breaking change to a model with an enforced contract during state comparison, it will now raise an error for versioned models and a warning for models that are not versioned. - **[Set `access` as a config](/reference/resource-configs/access):** It's now possible to set a model's `access` within config blocks in the model's file or the `dbt_project.yml` for an entire subfolder at once. From a5f02896a7cc883e884ff5d4b6b4ae60e20d3648 Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Wed, 11 Oct 2023 18:44:26 -0400 Subject: [PATCH 6/8] Update website/docs/guides/migration/versions/00-upgrading-to-v1.7.md Co-authored-by: Leona B. Campbell <3880403+runleonarun@users.noreply.github.com> --- website/docs/guides/migration/versions/00-upgrading-to-v1.7.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md b/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md index fe2eccce972..24261885bd2 100644 --- a/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md +++ b/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md @@ -39,7 +39,7 @@ dbt Labs is committed to providing backward compatibility for all versions 1.x, dbt Core v1.5 introduced model governance which we're continuing to refine. v1.7 includes these additional features and functionality: -- **[Breaking change detection](/reference/resource-properties/versions) for models with contracts enforced:** When dbt detects a breaking change to a model with an enforced contract during state comparison, it will now raise an error for versioned models and a warning for models that are not versioned. +- **[Breaking change detection](/reference/resource-properties/versions#detecting-breaking-changes) for models with contracts enforced:** When dbt detects a breaking change to a model with an enforced contract during state comparison, it will now raise an error for versioned models and a warning for models that are not versioned. - **[Set `access` as a config](/reference/resource-configs/access):** It's now possible to set a model's `access` within config blocks in the model's file or the `dbt_project.yml` for an entire subfolder at once. - **[Type aliasing for model contracts](/reference/resource-configs/contract):** dbt will use each adapter's built-in type aliasing for user-provided data types—meaning you can now write `string` always, and dbt will translate to `text` on Postgres/Redshift. This is on by default, and it's possible to opt-out. - **[Raise warning for numeric types](/reference/resource-configs/contract):** This led to issues when putting `numeric` in model contracts without considering that Snowflake interprets it as `numeric(38,0)`, and will round decimals accordingly. dbt will raise a warning if it sees a numeric type without specified precision/scale. From b66dfac25612f79a7e4e478d65b39014c9c5993d Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Wed, 11 Oct 2023 18:46:12 -0400 Subject: [PATCH 7/8] Apply suggestions from code review Co-authored-by: Leona B. Campbell <3880403+runleonarun@users.noreply.github.com> --- .../guides/migration/versions/00-upgrading-to-v1.7.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md b/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md index 24261885bd2..fc60b573d71 100644 --- a/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md +++ b/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md @@ -40,12 +40,13 @@ dbt Labs is committed to providing backward compatibility for all versions 1.x, dbt Core v1.5 introduced model governance which we're continuing to refine. v1.7 includes these additional features and functionality: - **[Breaking change detection](/reference/resource-properties/versions#detecting-breaking-changes) for models with contracts enforced:** When dbt detects a breaking change to a model with an enforced contract during state comparison, it will now raise an error for versioned models and a warning for models that are not versioned. -- **[Set `access` as a config](/reference/resource-configs/access):** It's now possible to set a model's `access` within config blocks in the model's file or the `dbt_project.yml` for an entire subfolder at once. -- **[Type aliasing for model contracts](/reference/resource-configs/contract):** dbt will use each adapter's built-in type aliasing for user-provided data types—meaning you can now write `string` always, and dbt will translate to `text` on Postgres/Redshift. This is on by default, and it's possible to opt-out. -- **[Raise warning for numeric types](/reference/resource-configs/contract):** This led to issues when putting `numeric` in model contracts without considering that Snowflake interprets it as `numeric(38,0)`, and will round decimals accordingly. dbt will raise a warning if it sees a numeric type without specified precision/scale. +- **[Set `access` as a config](/reference/resource-configs/access):** You can now set a model's `access` within config blocks in the model's file or in the `dbt_project.yml` for an entire subfolder at once. +- **[Type aliasing for model contracts](/reference/resource-configs/contract):** dbt will use each adapter's built-in type aliasing for user-provided data types—meaning you can now write `string` always, and dbt will translate to `text` on Postgres/Redshift. This is "on" by default, but you can opt-out. +- **[Raise warning for numeric types](/reference/resource-configs/contract):** Because of issues when putting `numeric` in model contracts without considering that default values such as `numeric(38,0)` might round decimals accordingly. dbt will now warn you if it finds a numeric type without specified precision/scale. ### Quick hits -- You can configure a `delimiter` for a seed file. -- Support packages with the same git repo and unique subdirectory. +With these quick hits, you can now: +- Configure a `delimiter` for a seed file. +- Use packages with the same git repo and unique subdirectory. - Moved the `date_spine` macro from dbt-utils to dbt-core. From 65fa0c6f266fb2eac05d04138b747148dbad0ba8 Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Wed, 11 Oct 2023 18:46:27 -0400 Subject: [PATCH 8/8] Apply suggestions from code review --- website/docs/guides/migration/versions/00-upgrading-to-v1.7.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md b/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md index fc60b573d71..ba079e6a0fb 100644 --- a/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md +++ b/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md @@ -47,6 +47,6 @@ dbt Core v1.5 introduced model governance which we're continuing to refine. v1. ### Quick hits With these quick hits, you can now: -- Configure a `delimiter` for a seed file. +- Configure a `delimiter` for a seed file. - Use packages with the same git repo and unique subdirectory. - Moved the `date_spine` macro from dbt-utils to dbt-core.