Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

connector: allow customization on the prefix of group.id for Kafka readers #13386

Closed
BugenZhao opened this issue Nov 13, 2023 · 3 comments
Closed
Labels
no-issue-activity type/enhancement Improvements to existing implementation.

Comments

@BugenZhao
Copy link
Member

Currently we hardcode it to be consumer-. Users may want to customize it to be consumer_ or something else.

if config.get("group.id").is_none() {
config.set(
"group.id",
format!(
"consumer-{}",
SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap()
.as_micros()
),
);
}

Note that specifying the whole group id is not expected as described in #13223 (comment). So maybe we should remove consumer_group from properties to avoid confusion.

cc @shanicky @tabVersion

@BugenZhao BugenZhao added the type/enhancement Improvements to existing implementation. label Nov 13, 2023
@github-actions github-actions bot added this to the release-1.5 milestone Nov 13, 2023
@tabVersion
Copy link
Contributor

So maybe we should remove consumer_group from properties to avoid confusion.

We've removed the group.id from the dev doc to avoid unexpected behavior.

Users may want to customize it to be consumer_ or something else.

I am afraid of bringing up more confusion for normal users that we have to explain how RW uses the low-level API and how splits are distributed to different actors. group.id is required by the kafka client but we don't necessarily need it.

@BugenZhao BugenZhao removed this from the release-1.5 milestone Dec 6, 2023
Copy link
Contributor

This issue has been open for 60 days with no activity. Could you please update the status? Feel free to continue discussion or close as not planned.

@xxchan
Copy link
Member

xxchan commented Nov 8, 2024

#17657

@xxchan xxchan closed this as completed Nov 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
no-issue-activity type/enhancement Improvements to existing implementation.
Projects
None yet
Development

No branches or pull requests

3 participants