Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use correct join node specs for GroupByMetrics #1193

Merged
merged 9 commits into from
May 10, 2024
Merged

Use correct join node specs for GroupByMetrics #1193

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
14c6d61
Remove LinkableMetric options that require specifying an outer join path
courtneyholcomb May 7, 2024
35d86bf
Move logic for building group by metric source nodes so they don't ge…
courtneyholcomb May 7, 2024
76540ae
Update test to check ALL available filters (not just single-hop)
courtneyholcomb May 7, 2024
187a922
Rename is_aggregated_to_elements to just aggregated_to_elements
courtneyholcomb May 8, 2024
ce861fe
Track full specs instead of just references in aggregated_to_elements…
courtneyholcomb May 8, 2024
1633e0a
Update logic determining specs to include in joined node for group by…
courtneyholcomb May 8, 2024
2cc544a
Update errors to help with debugging + remove print
courtneyholcomb May 8, 2024
e39de1c
Add variable for readability
courtneyholcomb May 10, 2024
f3b9c14
Delete accidental snapshots
courtneyholcomb May 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions metricflow/dataset/sql_dataset.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,11 +74,11 @@ def column_associations_for_entity(

if len(matching_instances) != 1:
raise RuntimeError(
f"Expected exactly one matching instance for {entity_spec} in instance set, but found: {matching_instances}"
f"Expected exactly one matching instance for {entity_spec} in instance set, but found: {matching_instances}. "
f"All entity instances: {self.instance_set.entity_instances}"
)
matching_instance = matching_instances[0]
if not matching_instance.associated_columns:
print("entity links to compare:", entity_spec.entity_links, linkable_instance.spec.entity_links)
raise RuntimeError(
f"No associated columns for entity instance {matching_instance} in data set."
"This indicates internal misconfiguration."
Expand Down
22 changes: 14 additions & 8 deletions metricflow/plan_conversion/instance_converters.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -558,19 +558,25 @@ def transform(self, instance_set: InstanceSet) -> InstanceSet: # noqa: D102
# Sanity check to make sure the specs are in the instance set

if self._include_specs:
include_specs_not_found = []
for include_spec in self._include_specs.all_specs:
if include_spec not in instance_set.spec_set.all_specs:
raise RuntimeError(
f"Include spec {include_spec} is not in the spec set {instance_set.spec_set} - "
f"check if this node was constructed correctly."
)
include_specs_not_found.append(include_spec)
if include_specs_not_found:
raise RuntimeError(
f"Include specs {include_specs_not_found} are not in the spec set {instance_set.spec_set} - "
f"check if this node was constructed correctly."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this is an odd follow up. If I think it was constructed correctly and I hit this error, what does that indicate?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It indicates that you either 1) constructed the instance set correctly or 2) constructed the node incorrectly, since they don't match!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was related to this PR because I ran into this error and that's why I needed to change the instance specs built in the JoinDescription. Definitely could have been split into another PR though.

)
elif self._exclude_specs:
exclude_specs_not_found = []
for exclude_spec in self._exclude_specs.all_specs:
if exclude_spec not in instance_set.spec_set.all_specs:
raise RuntimeError(
f"Exclude spec {exclude_spec} is not in the spec set {instance_set.spec_set} - "
f"check if this node was constructed correctly."
)
exclude_specs_not_found.append(exclude_spec)
if exclude_specs_not_found:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is very nice to get all of these at once. 😁

raise RuntimeError(
f"Exclude specs {exclude_specs_not_found} are not in the spec set {instance_set.spec_set} - "
f"check if this node was constructed correctly."
)
else:
assert False, "Include specs or exclude specs should have been specified."

Expand Down
Loading