Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(storage): fix leaving put key after delete keys are dropped cause inconsistent #15232

Merged
merged 2 commits into from
Feb 23, 2024

Conversation

Little-Wallace
Copy link
Contributor

@Little-Wallace Little-Wallace commented Feb 23, 2024

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

fix #15057 (comment)

When we put some keys and then delete them in one epoch, if hummock enable "spill memtable", and then compact them in the last level of LSM tree, compactor may lost DELETE keys when and only when the safe watermark keeps same with just flushed epoch. We used to assume that the epoch of current keys must be larger than the follower keys, but it is wrong since we start using pure epoch to decide whether it shall be dropped.

So compactor drops the delete keys and left the put key because they share the same pure_epoch and it exactly equals task_config.watermark.

Checklist

  • I have written necessary rustdoc comments
  • I have added necessary unit tests and integration tests
  • I have added test labels as necessary. See details.
  • I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
  • My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
  • All checks passed in ./risedev check (or alias, ./risedev c)
  • My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)
  • My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

  • My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note

If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.

Signed-off-by: Little-Wallace <[email protected]>
@github-actions github-actions bot added the type/fix Bug fix label Feb 23, 2024
@Little-Wallace Little-Wallace added this pull request to the merge queue Feb 23, 2024
Merged via the queue into main with commit e19aaec Feb 23, 2024
30 of 31 checks passed
@Little-Wallace Little-Wallace deleted the wallace/fix-repeated branch February 23, 2024 17:19
github-actions bot pushed a commit that referenced this pull request Feb 23, 2024
hzxa21 pushed a commit that referenced this pull request Feb 23, 2024
… inconsistent (#15232) (#15233)

Signed-off-by: Little-Wallace <[email protected]>
Co-authored-by: Wallace <[email protected]>
zwang28 pushed a commit that referenced this pull request Feb 24, 2024
@wcy-fdu wcy-fdu mentioned this pull request Feb 26, 2024
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/fix Bug fix
Projects
None yet
Development

Successfully merging this pull request may close these issues.

PG cdc q3 checksums inconsistent
3 participants