Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(streaming): non-blocking join matching #13776

Open
BugenZhao opened this issue Dec 4, 2023 · 3 comments
Open

perf(streaming): non-blocking join matching #13776

BugenZhao opened this issue Dec 4, 2023 · 3 comments

Comments

@BugenZhao
Copy link
Member

...to further reduce the memory usage and also the latency, I guess we can even make hash_eq_match return a Stream instead of collecting the rows into an entry before we can yield them.

Originally posted by @BugenZhao in #10979 (comment)

@github-actions github-actions bot added this to the release-1.5 milestone Dec 4, 2023
@BugenZhao BugenZhao added component/streaming Stream processing related issue. type/perf labels Dec 4, 2023
@fuyufjh fuyufjh modified the milestones: release-1.5, release-1.6 Dec 4, 2023
@fuyufjh fuyufjh removed this from the release-1.6 milestone Jan 9, 2024
@lmatz
Copy link
Contributor

lmatz commented Jan 22, 2024

This is supposed to prevent the OOM case where a single row from the dimension table produces a huge number of joined results, e.g. 1M+, with the fact table, isn't it? (it occurred to me when looking at today's user OOM feedback

@BugenZhao
Copy link
Member Author

This is supposed to prevent the OOM case where a single row from the dimension table produces a huge number of joined results, e.g. 1M+, with the fact table, isn't it? (it occurred to me when looking at today's user OOM feedback

I believe so. Work together with #10979.

Copy link
Contributor

This issue has been open for 60 days with no activity. Could you please update the status? Feel free to continue discussion or close as not planned.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants