Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug fix: custom granularity join column gets pruned #1425

Merged
merged 2 commits into from
Sep 26, 2024

Conversation

courtneyholcomb
Copy link
Contributor

@courtneyholcomb courtneyholcomb commented Sep 24, 2024

We have an issue with the SqlColumnPruner where it might prune a column that's used in the join condition, if it's not used anywhere else. There isn't a simple way to fix that issue in the optimizer itself, since the join condition can contain any SqlExpressionNode, and this appears to only be relevant for the new custom granularity node. Thus, I've fixed this in the custom granularity node instead of the optimizer by rendering the column's expression in the join condition instead of its alias.

@cla-bot cla-bot bot added the cla:yes label Sep 24, 2024
@courtneyholcomb courtneyholcomb changed the title Court/custom grain where 1 Support custom grain in where filters Sep 24, 2024
@dbt-labs dbt-labs deleted a comment from github-actions bot Sep 24, 2024
@courtneyholcomb courtneyholcomb changed the title Support custom grain in where filters Bug fix: custom granularity join column gets pruned Sep 24, 2024
@courtneyholcomb courtneyholcomb force-pushed the court/custom-grain-where-1 branch from 4da3888 to 5ef7eb6 Compare September 24, 2024 22:32
@courtneyholcomb courtneyholcomb marked this pull request as ready for review September 24, 2024 22:36
@courtneyholcomb courtneyholcomb added the Run Tests With Other SQL Engines Runs the test suite against the SQL engines in our target environment label Sep 24, 2024
@github-actions github-actions bot removed the Run Tests With Other SQL Engines Runs the test suite against the SQL engines in our target environment label Sep 24, 2024
Copy link
Contributor

@WilliamDee WilliamDee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did a pass, this looks great to me 🥳 would love for another reviewer to just do a pass as well tho

metricflow/plan_conversion/dataflow_to_sql.py Outdated Show resolved Hide resolved
metricflow/dataflow/nodes/join_to_custom_granularity.py Outdated Show resolved Hide resolved
metric_specs=(MetricSpec("listings"),),
time_dimension_specs=(
metric_time_with_custom_grain,
TimeDimensionSpec(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think we need a test that has martian_day and metric_time__day since that is what is used for the time spine join? Or does this case sufficiently test that already

@courtneyholcomb courtneyholcomb force-pushed the court/custom-grain-where-1 branch from 5ff504b to f52621f Compare September 25, 2024 19:14
Base automatically changed from court/parse-custom-4 to main September 26, 2024 01:19
@courtneyholcomb courtneyholcomb force-pushed the court/custom-grain-where-1 branch from f52621f to e0ea989 Compare September 26, 2024 01:20
@courtneyholcomb courtneyholcomb enabled auto-merge (squash) September 26, 2024 01:20
@courtneyholcomb courtneyholcomb merged commit ebee1ad into main Sep 26, 2024
15 checks passed
@courtneyholcomb courtneyholcomb deleted the court/custom-grain-where-1 branch September 26, 2024 01:22
courtneyholcomb added a commit that referenced this pull request Sep 26, 2024
Implements some small PR feedback from
#1425.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants