fix(storage): fix the trivial-move loop caused by config and pick_whole_level #17721
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
zwang28
added
the
need-cherry-pick-release-1.10
zwang28
approved these changes
Jul 17, 2024
| self.config.max_l0_compact_level_count as usize, | ||
| self.config | ||
| .max_l0_compact_level_count | ||
| .unwrap_or(compaction_config::max_l0_compact_level_count()) as usize, |
There was a problem hiding this comment.
reminder: compaction_config::max_l0_compact_level_count() makes sense only because it's not configurable in toml config file. Otherwise we should use the value from config file rather than the static default one.
Was the omission of max_l0_compact_level_count from the config file unintentional?
There was a problem hiding this comment.
Its not omission, we change the setting of max_l0_compact_level_count to Option and the default value is None (So it's invisible.)
There was a problem hiding this comment.
I mean it's not included here
github-merge-queue bot
pushed a commit
that referenced
this pull request
Jul 17, 2024
…le_level (#17721) (#17723) Co-authored-by: Li0k <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
Previously, we stored
max_l0_compact_level_countas a config in CompactionConfig, and the default value will change to 0 after upgrading the old cluster.And
max_l0_compact_level_count = 0would cause the task to be passed over by the validator, which triggered a bug in thepick_whole_levelfunction.pick_whole_levelpicks out the tasks that contain only one sublevel, which triggers the in-place trivial-move, and the meta gets bogged down.This PR
max_l0_compact_level_countpick_whole_levelChecklist
./risedev check(or alias,./risedev c)Documentation
Release note
If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.