Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Push resolved linkable elements through to WhereFilterSpec #1179

Merged
merged 2 commits into from
May 7, 2024

Conversation

tlento
Copy link
Contributor

@tlento tlento commented May 3, 2024

Push resolved linkable elements through to WhereFilterSpec

The resolved LinkableElements gathered during the filter spec
resolution phase of query processing need to be available to
each WhereFilterSpec so that we can determine whether or not
a given WhereFilterSpec is eligible for predicate pushdown.

This change simply adds the necessary property and wires these
elements through to where we will need to access them in the
DataflowPlanBuilder. This change proved a bit troublesome due
to the SerializableDataclass construct, which has some particular
requirements which required some slightly odd inheritance tagging.

Copy link

github-actions bot commented May 3, 2024

Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide.

Copy link
Contributor

@courtneyholcomb courtneyholcomb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@@ -672,6 +677,7 @@ class WhereFilterSpec(Mergeable, SerializableDataclass):
where_sql: str
bind_parameters: SqlBindParameters
linkable_specs: Tuple[LinkableInstanceSpec, ...]
linkable_elements: Tuple[LinkableElement, ...]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not necessary for this PR, but I wonder if it would be cleaner to add some simple converters between linkable specs and linkable elements so that you don't have to pass both through everywhere.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've talked about consolidating these and I think we'll get there eventually. One of the issues we have right now is LinkableSpecs aren't convertible to LinkableElements because the latter contain a bunch of additional metadata that LinkableSpecs can't carry along with them, so there's no way to simply add that information without causing all kinds of havoc with existing spec comparisons (e.g., some_entity_spec == some_other_entity_spec).

For now I could pass the LinkableElementSet along but that gets slightly more complicated so I decided to wire the elements through. I might still switch over to passing the set to the end here. I'm honestly not sure which approach is less annoying to read.

Copy link
Contributor Author

@tlento tlento May 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could replace the linkable_specs with a property accessor that converts from the element to a spec, I suppose, but it's a bit gnarly without the LinkableElementSet.

@tlento tlento force-pushed the linkable-elements-to-filter-spec-lookup branch from 4a62cec to c2e773b Compare May 6, 2024 17:29
@tlento tlento force-pushed the linkable-elements-to-where-filter-spec branch from 402f25f to db53b83 Compare May 6, 2024 17:29
@tlento tlento force-pushed the linkable-elements-to-filter-spec-lookup branch from c2e773b to 3f454a4 Compare May 6, 2024 22:46
@tlento tlento force-pushed the linkable-elements-to-where-filter-spec branch from db53b83 to c494b04 Compare May 6, 2024 22:46
@tlento tlento force-pushed the linkable-elements-to-filter-spec-lookup branch from 3f454a4 to 73f7b85 Compare May 7, 2024 00:47
Base automatically changed from linkable-elements-to-filter-spec-lookup to main May 7, 2024 01:01
tlento added 2 commits May 6, 2024 18:09
The resolved LinkableElements gathered during the filter spec
resolution phase of query processing need to be available to
each WhereFilterSpec so that we can determine whether or not
a given WhereFilterSpec is eligible for predicate pushdown.

This change simply adds the necessary property and wires these
elements through to where we will need to access them in the
DataflowPlanBuilder. This change proved a bit troublesome due
to the SerializableDataclass construct, which has some particular
requirements which required some slightly odd inheritance tagging.
@tlento tlento force-pushed the linkable-elements-to-where-filter-spec branch from c494b04 to 88469f0 Compare May 7, 2024 01:17
@tlento tlento merged commit 62652c2 into main May 7, 2024
24 checks passed
@tlento tlento deleted the linkable-elements-to-where-filter-spec branch May 7, 2024 01:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants