-
Notifications
You must be signed in to change notification settings - Fork 591
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: Fix the panic during parallel unit mapping conversion. #17318
Merged
shanicky
merged 6 commits into
main
from
17288-meta-node-panicked-at-srccommonsrchashconsistent_hashmappingrs26453-no-entry-found-for-key
Jun 20, 2024
Merged
fix: Fix the panic during parallel unit mapping conversion. #17318
shanicky
merged 6 commits into
main
from
17288-meta-node-panicked-at-srccommonsrchashconsistent_hashmappingrs26453-no-entry-found-for-key
Jun 20, 2024
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ked-at-srccommonsrchashconsistent_hashmappingrs26453-no-entry-found-for-key
yezizp2012
approved these changes
Jun 20, 2024
shanicky
added
need-cherry-pick-release-1.9
need-cherry-pick-release-1.10
Open a cherry-pick PR to branch release-1.10 after the current PR is merged
labels
Jun 20, 2024
…consistent_hashmappingrs26453-no-entry-found-for-key
shanicky
deleted the
17288-meta-node-panicked-at-srccommonsrchashconsistent_hashmappingrs26453-no-entry-found-for-key
branch
June 20, 2024 08:19
github-actions bot
pushed a commit
that referenced
this pull request
Jun 20, 2024
Co-authored-by: Shanicky Chen <>
github-merge-queue bot
pushed a commit
that referenced
this pull request
Jun 20, 2024
…17367) Co-authored-by: Shanicky Chen <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
ci/run-e2e-single-node-tests
need-cherry-pick-release-1.10
Open a cherry-pick PR to branch release-1.10 after the current PR is merged
type/fix
Bug fix
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
Title:
Refactor consistent hashing and improve error handling in metadata controllers
Summary:
This pull request introduces a series of updates to Rust files associated with consistent hashing and metadata controllers in our distributed system. Below is an overview of the main changes incorporated into the system:
Changes to
consistent_hash/masking.rs
:HashSet
import fromstd::collections
Sub
import fromstd::ops
ParallelUnitError
to handle un-mapped parallel units in the systemParallelUnitMapping::to_worker_slot
to return aResult
type, now checking complete coverage of parallel unit IDs by worker slots and returning an error if uncoveredas_delete_worker_slap_mapping
toParallelUnitMapping
for creating aWorkerSlotMapping
indicating delete operations via maxu64
valuesModifications to
catalog.rs
:ParallelUnitMapping
importdrop_database
,deploy_stream_job
,stop_stream_job
) to set the worker slot mapping toNone
inPbFragmentWorkerSlotMapping
instead of computing a mapping indicating a shift in mapping handlingget_parallel_unit_to_worker_map
function from several methodsUpdates to
fragment.rs
andstreaming_job.rs
:ParallelUnitMapping::to_worker_slot
now returning aResult
, enabling proper error propagationconvert_fragment_mappings
infragment',' which converts fragment mappings to protobuf representations while handling errors from
to_worker_slot`The overarching goal of these changes is to streamline and enhance error handling for converting parallel unit mappings to worker slot mappings, which is a pivotal aspect of our metadata controllers. Affected code segments now anticipate potential conversion errors and are equipped for their management, markedly improving system robustness in handling parallel unit mapping scenarios.
Note: Reviewers should carefully assess the potential impacts of setting the worker slot mappings to
None
and confirm that the propagation of conversion errors aligns with the intended behavior of the distributed system architecture.Checklist
./risedev check
(or alias,./risedev c
)