Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: improve non-append-only temporal join for non-join key update #16378

Open
chenzl25 opened this issue Apr 18, 2024 · 2 comments
Open

feat: improve non-append-only temporal join for non-join key update #16378

chenzl25 opened this issue Apr 18, 2024 · 2 comments
Assignees

Comments

@chenzl25
Copy link
Contributor

chenzl25 commented Apr 18, 2024

I think we need to handle the case that a row is updated while the join key is not changed. This can be done in a separate PR, but overall I think it's a must-have enhancement

Originally posted by @fuyufjh in #16286 (comment)

@github-actions github-actions bot added this to the release-1.9 milestone Apr 18, 2024
@chenzl25
Copy link
Contributor Author

@fuyufjh I just realized that the case of a row is updated while the join key is not changed couldn't be optimized because we can't know the inner table whether has been updated. If the inner table is updated we need to delete the row from memo table and insert a row into the memo table with the latest inner table's values.

@fuyufjh
Copy link
Member

fuyufjh commented May 16, 2024

@fuyufjh I just realized that the case of a row is updated while the join key is not changed couldn't be optimized because we can't know the inner table whether has been updated. If the inner table is updated we need to delete the row from memo table and insert a row into the memo table with the latest inner table's values.

I think this behavior is up to us to define. That is, when a row changed while the join key in that row didn't change, either updating the result or not makes sense.

Considering the users chose temporal join instead of normal join, I suppose he majorly cares about the changes from outer side (fact table) rather than inner side (dim table). From this perspective, I tend to not updating the result under this case.

@chenzl25 chenzl25 removed this from the release-2.1 milestone Oct 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants