Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

rdkafka consumes too much memory during longevity test and OOM #13493

Closed
lmatz opened this issue Nov 17, 2023 · 3 comments
Closed

rdkafka consumes too much memory during longevity test and OOM #13493

lmatz opened this issue Nov 17, 2023 · 3 comments
Labels
found-by-longevity-test out-of-memory type/bug Something isn't working

Comments

@lmatz
Copy link
Contributor

lmatz commented Nov 17, 2023 •
edited
Loading

Just leave another data point for visibility if we want to test if the problem is solved later.

The following setting makes it easier to reproduce the problem:
https://buildkite.com/risingwave-test/longevity-test/builds/779#018bb714-096f-442e-905f-3af721bb056a
i.e. repeat the nexmark query 8 times and run them all together concurrently. 25*8=150 with only 1000 rows/s throughput.

Please take a look at the Environment tag in Buildkite when trying to reproduce.

SCR-20231116-ort

It is an inherent issue of rdkafka.
@lmatz lmatz added type/bug Something isn't working found-by-longevity-test labels Nov 17, 2023
@github-actions github-actions bot added this to the release-1.5 milestone Nov 17, 2023
@fuyufjh
Copy link
Member

fuyufjh commented Nov 17, 2023

Duplicated #13326 #11977

@fuyufjh fuyufjh removed this from the release-1.5 milestone Dec 6, 2023
@fuyufjh
Copy link
Member

fuyufjh commented Dec 6, 2023

Kept as known issue. Removed from milestone.

@lmatz
Copy link
Contributor Author

lmatz commented Jan 9, 2024

will create new issue if happened again, bypass it by using a shared materialized view as the source table for all the MVs

@lmatz lmatz closed this as not planned Won't fix, can't repro, duplicate, stale Jan 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
found-by-longevity-test out-of-memory type/bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lmatz @fuyufjh