Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add test time spines for sub-daily granularity #1358

Merged
merged 6 commits into from
Jul 30, 2024
+5,769 −2,050
Merged

Add test time spines for sub-daily granularity #1358

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
9cec4eb
Add test time spines for sub-daily granularity
courtneyholcomb Jul 27, 2024
d7ea994
Update snapshots - all node id changes due to new time spine source n…
courtneyholcomb Jul 27, 2024
fdcf983
Update time spine test fixture to support multiple
courtneyholcomb Jul 27, 2024
ed9555c
Update SQL engine snapshots
courtneyholcomb Jul 29, 2024
2ef8de8
Remove tests that require engine-specific syntax in metric definitions
courtneyholcomb Jul 29, 2024
ae263fd
Handle no-metric queries with sub-daily `metric_time` (#1359)
courtneyholcomb Jul 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 0 additions & 19 deletions ...cs/metricflow_semantics/test_helpers/semantic_manifest_yamls/simple_manifest/metrics.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -783,22 +783,3 @@ metric:
name: listings
filter: "{{ Metric('views', ['listing']) }} > 10"
time_granularity: week
---
metric:
name: bookings_before_dec_20_2019
description: Bookings up to the end of 2022
type: simple
type_params:
measure: bookings
filter: "{{ TimeDimension('metric_time') }} < '2012-12-20'"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I think I know why this is. Timestamp literals in Trino have to take the form TIMESTAMP <literal> so we'd need to do some substitution somewhere.

If the error is due to the filter it means we have to do custom rendering against the filter expr.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep. I couldn't come up with an easy way to fix it here and wasn't sure it was worth doing the hard fix for an engine we barely use.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. I think the right way to deal with this is via a more holistic approach to filter expression inputs. In the meantime, having a small gap in Trino test coverage - particularly one where the main difference essentially boils down to how we go about pasting in the user-provided expression at render time - seems fine to me.

---
metric:
name: bookings_between_dec_18_2019_and_dec_20_2019
description: Bookings starting in 2020. Used to test a metric with different types of ambiguous filters in on its input metric.
type: derived
type_params:
expr: bookings_before_dec_20_2019
metrics:
- name: bookings_before_dec_20_2019
filter: "{{ TimeDimension('metric_time') }} > '2019-12-18'"
time_granularity: week
6 changes: 0 additions & 6 deletions ...shots/test_linkable_spec_resolver.py/list/test_linkable_element_set_as_spec_set__set0.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,6 @@
'listing__booking__listing__booking_value_sub_instant',
'listing__booking__listing__booking_value_sub_instant_add_10',
'listing__booking__listing__bookings',
'listing__booking__listing__bookings_before_dec_20_2019',
'listing__booking__listing__bookings_between_dec_18_2019_and_dec_20_2019',
'listing__booking__listing__bookings_fill_nulls_with_0',
'listing__booking__listing__bookings_fill_nulls_with_0_without_time_spine',
'listing__booking__listing__bookings_join_to_time_spine',
Expand Down Expand Up @@ -56,8 +54,6 @@
'listing__booking_value_sub_instant',
'listing__booking_value_sub_instant_add_10',
'listing__bookings',
'listing__bookings_before_dec_20_2019',
'listing__bookings_between_dec_18_2019_and_dec_20_2019',
'listing__bookings_fill_nulls_with_0',
'listing__bookings_fill_nulls_with_0_without_time_spine',
'listing__bookings_join_to_time_spine',
Expand Down Expand Up @@ -266,8 +262,6 @@
'user__listing__user__booking_value_sub_instant',
'user__listing__user__booking_value_sub_instant_add_10',
'user__listing__user__bookings',
'user__listing__user__bookings_before_dec_20_2019',
'user__listing__user__bookings_between_dec_18_2019_and_dec_20_2019',
'user__listing__user__bookings_fill_nulls_with_0',
'user__listing__user__bookings_fill_nulls_with_0_without_time_spine',
'user__listing__user__bookings_join_to_time_spine',
Expand Down
6 changes: 0 additions & 6 deletions ...hots/test_semantic_model_container.py/str/test_linkable_elements_for_measure__result0.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,6 @@ Model Join-Path Entity Links
('listings_latest',) ("('listing',)", "('booking', 'listing')") booking_value_sub_instant ['JOINED', 'METRIC']
('listings_latest',) ("('listing',)", "('booking', 'listing')") booking_value_sub_instant_add_10 ['JOINED', 'METRIC']
('listings_latest',) ("('listing',)", "('booking', 'listing')") bookings ['JOINED', 'METRIC']
('listings_latest',) ("('listing',)", "('booking', 'listing')") bookings_before_dec_20_2019 ['JOINED', 'METRIC']
('listings_latest',) ("('listing',)", "('booking', 'listing')") bookings_between_dec_18_2019_and_dec_20_2019 ['JOINED', 'METRIC']
('listings_latest',) ("('listing',)", "('booking', 'listing')") bookings_fill_nulls_with_0 ['JOINED', 'METRIC']
('listings_latest',) ("('listing',)", "('booking', 'listing')") bookings_fill_nulls_with_0_without_time_spine ['JOINED', 'METRIC']
('listings_latest',) ("('listing',)", "('booking', 'listing')") bookings_join_to_time_spine ['JOINED', 'METRIC']
Expand Down Expand Up @@ -79,8 +77,6 @@ Model Join-Path Entity Links
('listings_latest',) ("('listing',)", "('listing',)") booking_value_sub_instant ['JOINED', 'METRIC']
('listings_latest',) ("('listing',)", "('listing',)") booking_value_sub_instant_add_10 ['JOINED', 'METRIC']
('listings_latest',) ("('listing',)", "('listing',)") bookings ['JOINED', 'METRIC']
('listings_latest',) ("('listing',)", "('listing',)") bookings_before_dec_20_2019 ['JOINED', 'METRIC']
('listings_latest',) ("('listing',)", "('listing',)") bookings_between_dec_18_2019_and_dec_20_2019 ['JOINED', 'METRIC']
('listings_latest',) ("('listing',)", "('listing',)") bookings_fill_nulls_with_0 ['JOINED', 'METRIC']
('listings_latest',) ("('listing',)", "('listing',)") bookings_fill_nulls_with_0_without_time_spine ['JOINED', 'METRIC']
('listings_latest',) ("('listing',)", "('listing',)") bookings_join_to_time_spine ['JOINED', 'METRIC']
Expand Down Expand Up @@ -140,8 +136,6 @@ Model Join-Path Entity Links
('listings_latest',) ("('user',)", "('listing', 'user')") booking_value_sub_instant ['JOINED', 'METRIC']
('listings_latest',) ("('user',)", "('listing', 'user')") booking_value_sub_instant_add_10 ['JOINED', 'METRIC']
('listings_latest',) ("('user',)", "('listing', 'user')") bookings ['JOINED', 'METRIC']
('listings_latest',) ("('user',)", "('listing', 'user')") bookings_before_dec_20_2019 ['JOINED', 'METRIC']
('listings_latest',) ("('user',)", "('listing', 'user')") bookings_between_dec_18_2019_and_dec_20_2019 ['JOINED', 'METRIC']
('listings_latest',) ("('user',)", "('listing', 'user')") bookings_fill_nulls_with_0 ['JOINED', 'METRIC']
('listings_latest',) ("('user',)", "('listing', 'user')") bookings_fill_nulls_with_0_without_time_spine ['JOINED', 'METRIC']
('listings_latest',) ("('user',)", "('listing', 'user')") bookings_join_to_time_spine ['JOINED', 'METRIC']
Expand Down
18 changes: 0 additions & 18 deletions ...emantic_model_container.py/tuple/test_linkable_elements_for_no_metrics_query__result0.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,8 +59,6 @@
'company__listing__user__company__booking_value_sub_instant',
'company__listing__user__company__booking_value_sub_instant_add_10',
'company__listing__user__company__bookings',
'company__listing__user__company__bookings_before_dec_20_2019',
'company__listing__user__company__bookings_between_dec_18_2019_and_dec_20_2019',
'company__listing__user__company__bookings_fill_nulls_with_0',
'company__listing__user__company__bookings_fill_nulls_with_0_without_time_spine',
'company__listing__user__company__bookings_join_to_time_spine',
Expand Down Expand Up @@ -127,8 +125,6 @@
'guest__booking__guest__booking_value_sub_instant',
'guest__booking__guest__booking_value_sub_instant_add_10',
'guest__booking__guest__bookings',
'guest__booking__guest__bookings_before_dec_20_2019',
'guest__booking__guest__bookings_between_dec_18_2019_and_dec_20_2019',
'guest__booking__guest__bookings_fill_nulls_with_0',
'guest__booking__guest__bookings_fill_nulls_with_0_without_time_spine',
'guest__booking__guest__bookings_join_to_time_spine',
Expand Down Expand Up @@ -162,8 +158,6 @@
'guest__booking_value_sub_instant',
'guest__booking_value_sub_instant_add_10',
'guest__bookings',
'guest__bookings_before_dec_20_2019',
'guest__bookings_between_dec_18_2019_and_dec_20_2019',
'guest__bookings_fill_nulls_with_0',
'guest__bookings_fill_nulls_with_0_without_time_spine',
'guest__bookings_join_to_time_spine',
Expand Down Expand Up @@ -208,8 +202,6 @@
'host__booking__host__booking_value_sub_instant',
'host__booking__host__booking_value_sub_instant_add_10',
'host__booking__host__bookings',
'host__booking__host__bookings_before_dec_20_2019',
'host__booking__host__bookings_between_dec_18_2019_and_dec_20_2019',
'host__booking__host__bookings_fill_nulls_with_0',
'host__booking__host__bookings_fill_nulls_with_0_without_time_spine',
'host__booking__host__bookings_join_to_time_spine',
Expand Down Expand Up @@ -243,8 +235,6 @@
'host__booking_value_sub_instant',
'host__booking_value_sub_instant_add_10',
'host__bookings',
'host__bookings_before_dec_20_2019',
'host__bookings_between_dec_18_2019_and_dec_20_2019',
'host__bookings_fill_nulls_with_0',
'host__bookings_fill_nulls_with_0_without_time_spine',
'host__bookings_join_to_time_spine',
Expand Down Expand Up @@ -290,8 +280,6 @@
'listing__booking__listing__booking_value_sub_instant',
'listing__booking__listing__booking_value_sub_instant_add_10',
'listing__booking__listing__bookings',
'listing__booking__listing__bookings_before_dec_20_2019',
'listing__booking__listing__bookings_between_dec_18_2019_and_dec_20_2019',
'listing__booking__listing__bookings_fill_nulls_with_0',
'listing__booking__listing__bookings_fill_nulls_with_0_without_time_spine',
'listing__booking__listing__bookings_join_to_time_spine',
Expand Down Expand Up @@ -326,8 +314,6 @@
'listing__booking_value_sub_instant',
'listing__booking_value_sub_instant_add_10',
'listing__bookings',
'listing__bookings_before_dec_20_2019',
'listing__bookings_between_dec_18_2019_and_dec_20_2019',
'listing__bookings_fill_nulls_with_0',
'listing__bookings_fill_nulls_with_0_without_time_spine',
'listing__bookings_join_to_time_spine',
Expand Down Expand Up @@ -399,8 +385,6 @@
'lux_listing__listing__lux_listing__booking_value_sub_instant',
'lux_listing__listing__lux_listing__booking_value_sub_instant_add_10',
'lux_listing__listing__lux_listing__bookings',
'lux_listing__listing__lux_listing__bookings_before_dec_20_2019',
'lux_listing__listing__lux_listing__bookings_between_dec_18_2019_and_dec_20_2019',
'lux_listing__listing__lux_listing__bookings_fill_nulls_with_0',
'lux_listing__listing__lux_listing__bookings_fill_nulls_with_0_without_time_spine',
'lux_listing__listing__lux_listing__bookings_join_to_time_spine',
Expand Down Expand Up @@ -512,8 +496,6 @@
'user__listing__user__booking_value_sub_instant',
'user__listing__user__booking_value_sub_instant_add_10',
'user__listing__user__bookings',
'user__listing__user__bookings_before_dec_20_2019',
'user__listing__user__bookings_between_dec_18_2019_and_dec_20_2019',
'user__listing__user__bookings_fill_nulls_with_0',
'user__listing__user__bookings_fill_nulls_with_0_without_time_spine',
'user__listing__user__bookings_join_to_time_spine',
Expand Down
12 changes: 0 additions & 12 deletions tests_metricflow/integration/test_cases/itest_metrics.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2215,15 +2215,3 @@ integration_test:
GROUP BY {{ render_date_trunc("ds", TimeGranularity.DAY) }}
) a ON b.ds = a.ds
WHERE {{ render_between_time_constraint("b.ds", "2020-01-03", "2020-01-03") }}
---
integration_test:
name: metric_time_filter_on_input_metric
description: Test a metric with a filter defined on its input metric.
model: SIMPLE_MODEL
metrics: ["bookings_between_dec_18_2019_and_dec_20_2019"]
check_query: |
SELECT
SUM(1) AS bookings_between_dec_18_2019_and_dec_20_2019
FROM {{ source_schema }}.fct_bookings
WHERE ({{ render_date_trunc("ds", TimeGranularity.DAY) }} < '2012-12-20')
AND ({{ render_date_trunc("ds", TimeGranularity.WEEK) }} > '2019-12-18')
Loading