Add date spine macros to core #8616
Conversation
Codecov ReportPatch coverage has no change and project coverage change:
Additional details and impacted files@@ Coverage Diff @@
## main #8616 +/- ##
==========================================
+ Coverage 86.61% 86.62% +0.01%
==========================================
Files 175 176 +1
Lines 25596 25671 +75
==========================================
+ Hits 22169 22237 +68
- Misses 3427 3434 +7
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
QMalcolm
force-pushed
the
qmalcolm--8172-add-date-spine-macros-to-core
branch
from
356e547 to
fa8369e
Compare
The macros added are - date_spine - get_intervals_between - generate_series - get_powers_of_two We're adding these to core because they are becoming more prevalently used with the increase usage in the semantic layer. Basically if you are using the semantic layer currently, then it is almost a requirement to use dbt-utils, which is undesireable given the SL is supported directly in core. The primary focus of this was to just add `date_spine`. However, because `date_spine` depends on other macros, these other macros were also moved.
QMalcolm
force-pushed
the
qmalcolm--8172-add-date-spine-macros-to-core
branch
2 times, most recently
from
7fa5715 to
be7ffb3
Compare
QMalcolm
added
dbt-docs
QMalcolm
changed the title
|
Is there a deprecation plan in place for these macros in dbt-utils? If the implementations of the two diverge this could lead to some surprising behaviour - especially if dbt-utils is installed and its date spine macros take precedence over the ones defined in core (I think that's how the precedence works based on docs) |
Yes! Once this is merged I'm going to open a ticket on dbt-labs/dbt-utils that either I or @dbeatty10 (I believe) will take on. I found the old deprecation function before its removal, and I talked with @joellabes that it's fine to bring back. |
There was a problem hiding this comment.
Thanks for moving this!!!
I remember there were conversations about moving dbt-utils into core. Are we still thinking of doing it? @dbeatty10
At this point in time, only planning on moving |
|
Opened a new issue in dbt-labs/docs.getdbt.com: dbt-labs/docs.getdbt.com#4124 |
|
@dbeatty10 do we need to make any updates to dbt_utils docs? Or are we ok here? |
|
@graciegoheen dbt-labs/dbt-utils#839 should also include any relevant update to the docs in dbt_utils, so added docs updates to the acceptance criteria of that issue. |
* Add `date_spine` macro (and macros it depends on) from dbt-utils to core The macros added are - date_spine - get_intervals_between - generate_series - get_powers_of_two We're adding these to core because they are becoming more prevalently used with the increase usage in the semantic layer. Basically if you are using the semantic layer currently, then it is almost a requirement to use dbt-utils, which is undesireable given the SL is supported directly in core. The primary focus of this was to just add `date_spine`. However, because `date_spine` depends on other macros, these other macros were also moved. * Add adapter tests for `get_powers_of_two` macro * Add adapter tests for `generate_series` macro * Add adapter tests for `get_intervals_between` macro * Add adapter tests for `date_spine` macro * Improve test fixture for `date_spine` macro to work with multiple adapters * Cast to types to date in fixture_date_spine when targeting redshift * Improve test fixture for `get_intervals_between` macro to work with multiple adapters * changie doc for adding date_spine macro
resolves #8172
Problem
The date_spine macro in dbt-utils is getting used more regularly due to the semantic layer requiring a date spine model to exist. As such we should start including it with dbt-core instead of requiring people to get dbt-utils to do it. Additionally, as #8319 suggests, we want to begin auto generating the metricflow date spine model when semantic objects exist but the required date spine model does not. In that vein, this is a required precursor.
Solution
Move the macros into dbt-core 🚀
Note
We've also created adapter tests. To ensure the macros work in the specific adapters we maintain, I've opened branches on dbt-snowflake, dbt-redshift, and dbt-bigquery to run the tests. And I've run the tests in those branches against this branch of core.
Checklist