-
Notifications
You must be signed in to change notification settings - Fork 98
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Consolidate metric constraints in DataflowPlanBuilder #393
Conversation
68ea691
to
5036dac
Compare
@@ -243,6 +235,9 @@ def _build_metrics_output_node( | |||
aggregated_measures_node=aggregated_measures_node, | |||
) | |||
) | |||
|
|||
assert len(compute_metrics_nodes) > 0, "ComputeMetricsNode was not properly constructed" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this assertion have existed before or is this now needed due to how the constraints are being bundled?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just a sanity check that shouldve existed before, something wouldve went wrong if len == 0
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, this looks less weird! I do think some added clarity around how we populate the spec constraint would be good to have before merge.
metricflow/query/query_parser.py
Outdated
@@ -376,8 +377,26 @@ def _parse_and_validate_query( | |||
metric_references=metric_references, time_dimension_specs=where_time_specs | |||
) | |||
|
|||
metric_specs: List[MetricSpec] = [] | |||
for metric_reference in base_metric_references: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmmm. It was a little hard to wrap my head around this.
The reason we do base_metric_references is because we only evaluate constraints on metrics at this point. The input_metrics to a DERIVED metric have the constraints pushed into the spec inside MetricSemantics.
It might be worth it to consolidate this further - right now we do top level spec construction in the query parser and upstream dependent spec construction inside MetricSemantics. But I also think that's likely opening up a can of worms, since MetricSemantics is in need of some restructuring itself.
So, I've convinced myself we should leave this here. But let's add a comment somewhere in this vicinity for why the base_metric_references are the things we care about here.
Also, I suggest initializing the local variable right above this block, it's not as important as putting it on line 233 makes it seem. We might even want to factor this all out into a helper method, then use the docstring to explain everything that's going on. I think I like that best.
5036dac
to
73485c6
Compare
73485c6
to
91f9f08
Compare
Context
#352 (comment)
Changes
Essentially in the DataflowPlanBuilder, given a MetricSpec
Currently, it doesn't seem like there's a use case of doing both of these other than to cause slight confusion. At runtime when building the query, we just pass a list of MetricSpec to the builder that was initialized by the metric names only. So to remove the redundancy here, in the query parser we should attach the constraint to the MetricSpec that gets passed to the builder so that we don't have to retrieve the metric constraint from the Metric object directly downstream.