Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature request: reuse group.id for kafka source #13855

Closed
tabVersion opened this issue Dec 7, 2023 · 0 comments · Fixed by #13856
Closed

feature request: reuse group.id for kafka source #13855

tabVersion opened this issue Dec 7, 2023 · 0 comments · Fixed by #13856

Comments

@tabVersion
Copy link
Contributor

Is your feature request related to a problem? Please describe.

in current impl, we new a group id every time a source actor is built, which leaves tons of useless group id.
RisingWave cannot clean them automatically because it uses admin API of kafka.

config.set(
"group.id",
format!(
"consumer-{}",
SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap()
.as_micros()
),
);

The requirement is that two actors cannot share the same group.id in an ongoing job. So we can use {fragment id}-{actor id} as an identifier, which makes it global unique.

as actors do not change frequently

Describe the solution you'd like

No response

Describe alternatives you've considered

No response

Additional context

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant