-
Notifications
You must be signed in to change notification settings - Fork 591
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(mysql-cdc): enable heartbeat message to keep the offset up to date with upstream #12868
Conversation
921142b
to
014b2ab
Compare
...isingwave-connector-test/src/test/java/com/risingwave/connector/source/SourceTestClient.java
Show resolved
Hide resolved
Codecov Report
@@ Coverage Diff @@
## main #12868 +/- ##
==========================================
+ Coverage 69.14% 69.25% +0.11%
==========================================
Files 1487 1489 +2
Lines 244648 245828 +1180
==========================================
+ Hits 169150 170246 +1096
- Misses 75498 75582 +84
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 17 files with indirect coverage changes 📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
…e with upstream (#12868)
…e with upstream (#12868) (#12886) Co-authored-by: StrikeW <[email protected]>
split_offset_mapping | ||
.entry(msg.split_id) | ||
.or_insert(msg.offset.clone()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess this should apply to all kinds of messages that have empty keys and payloads? 🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I expect this patch only affect direct cdc use cases, because I am not sure when it would break other connectors.
I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
Enable Debezium to emit heartbeat message which contains a binlog offset for each
heartbeat.interval.ms
, so that the persisted source offset can be updated even though the upstream table doesn't have any update for a long time.Tested locally.
related: #12753
Checklist
./risedev check
(or alias,./risedev c
)Documentation
Release note
If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.