feat(storage): emergency picker for write_limit #12183
Conversation
src/meta/src/hummock/compaction/picker/compaction_task_validator.rs
Outdated
Show resolved
Hide resolved
src/meta/src/hummock/manager/mod.rs
Outdated
| @@ -887,7 +886,7 @@ impl HummockManager { | |||
| .sum::<usize>(), | |||
| start_time.elapsed() | |||
| ); | |||
| } else if is_trivial_move && can_trivial_move { | |||
There was a problem hiding this comment.
Is it because all trivial move tasks are TaskType::Dynamic that we can remove this check?
There was a problem hiding this comment.
IMO, we don't actually need to disable trivial-move, regardless of the type of task?
…nto li0k/storage_picker_for_write_limit
…nto li0k/storage_picker_for_write_limit
…nto li0k/storage_picker_for_write_limit
…nto li0k/storage_picker_for_write_limit
| @@ -616,6 +617,50 @@ pub fn default_level_selector() -> Box<dyn LevelSelector> { | |||
| Box::<DynamicLevelSelector>::default() | |||
| } | |||
|
|
|||
| #[derive(Default)] | |||
| pub struct EmergencySelector {} | |||
There was a problem hiding this comment.
Please move to an independent file, this file is enough large....
There was a problem hiding this comment.
ok, I'm going to separate the implementation of the different selectors in a separate pr and put it in a independent file, like the picker
There was a problem hiding this comment.
LGTM.
Please help to clarify whether removal of can_trivial_move is OK. @Little-Wallace
I will add it back |
…nto li0k/storage_picker_for_write_limit
|
nexmark test q20 with 64 sub_level count limit with 128 sub_level count limit |
…nto li0k/storage_picker_for_write_limit
Li0k
changed the title
Codecov Report
@@ Coverage Diff @@
## main #12183 +/- ##
==========================================
- Coverage 69.79% 69.76% -0.03%
==========================================
Files 1423 1424 +1
Lines 236196 236247 +51
==========================================
- Hits 164859 164828 -31
- Misses 71337 71419 +82
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 5 files with indirect coverage changes 📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
This pr introduces a new emergency picker and enables it when the write limit is triggered. Simple logic to avoid live locking problems with write limits.
Checklist
./risedev check(or alias,./risedev c)Documentation
Release note
If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.