Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[wip] try to remove parallel unit from scheduling process with worker slot #16828

Closed
wants to merge 62 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
decfccf
Add WorkerMapping to protos; refactor to worker ID usage.
shanicky Apr 7, 2024
5947afb
Merge remote-tracking branch 'origin/main' into peng/fe-remove-pu-2
shanicky Apr 16, 2024
724d037
Refactor observer, notification, & catalog files for better structure…
shanicky Apr 16, 2024
0916ac8
Merge remote-tracking branch 'origin/main' into peng/fe-remove-pu
shanicky Apr 17, 2024
8f7f9b3
Merge remote-tracking branch 'origin/main' into peng/fe-remove-pu
shanicky Apr 18, 2024
e40d0c8
Merge remote-tracking branch 'origin/main' into peng/fe-remove-pu
shanicky Apr 22, 2024
b4673e6
Refactor parallel unit-worker map method to utils module.
shanicky Apr 22, 2024
b50031c
Merge remote-tracking branch 'origin/main' into peng/fe-remove-pu
shanicky Apr 22, 2024
3b48ac9
Deprecate fields in MetaSnapshot & reserve tags.
shanicky Apr 22, 2024
36937ac
Merge branch 'main' into peng/fe-remove-pu
shanicky Apr 24, 2024
6125cab
update proto
shanicky Apr 24, 2024
85facae
Enhanced consistent hashing, actor mapping, and scheduling with new s…
shanicky Apr 15, 2024
15aa35a
Renamed variables for clarity in `consistent_hash` and `stream_graph`…
shanicky Apr 16, 2024
286712e
Fix: Remove unused imports & refactor hash/stream in Rust codebase
shanicky Apr 16, 2024
52f3618
Refactor `mapping.rs`: Deduplicate and remove unused imports
shanicky Apr 24, 2024
6e2f7c4
Refactor vnode & stream mgr, better clarity & testing
shanicky Apr 24, 2024
617739d
Refactor `stream_manager`: improve `worker_slots` readability
shanicky Apr 24, 2024
1c30665
Add ParallelUnitMapping import in stream_manager.rs from risingwave_c…
shanicky Apr 24, 2024
b8604b3
Remove unused `ParallelUnitMapping` import from `stream_manager.rs`
shanicky Apr 25, 2024
5a6cf17
Fix namespace in vnode_mapping test
shanicky Apr 25, 2024
0c8b540
Merge remote-tracking branch 'origin/main' into peng/fe-remove-pu
shanicky May 6, 2024
ff4e3fa
Merge remote-tracking branch 'origin/main' into peng/fe-remove-pu
shanicky May 9, 2024
ca47e5b
Fixed typos, clean up imports and whitespace in Rust files
shanicky May 11, 2024
f107855
Merge remote-tracking branch 'origin/main' into peng/fe-remove-pu
shanicky May 11, 2024
b6e2700
Refactor serving.rs for efficient worker-to-vnode mapping, update imp…
shanicky May 11, 2024
e3de9ff
Merge remote-tracking branch 'origin/main' into peng/fe-remove-pu
shanicky May 13, 2024
78622ad
Merge remote-tracking branch 'origin/main' into peng/fe-remove-pu
May 14, 2024
bc2b8c5
Merge remote-tracking branch 'origin/main' into peng/fe-remove-pu
May 16, 2024
40c8a7e
Refactor protobufs, Rust types, rename worker mappings
May 17, 2024
ee15f5d
Update vnode placement tests and unique counts
May 17, 2024
3f4ac77
Remove assert from LocalLookupJoinExecBuilder
May 17, 2024
93cb568
Refactor WorkerMapping to WorkerSlotMapping in protobuf and Rust code
May 17, 2024
5a0c927
Update proto defs, error msg, rename vars, type changes
May 17, 2024
f54bb6c
Remove typecast for ROOT_TASK_* constants
May 17, 2024
95ded68
Merge remote-tracking branch 'origin/main' into peng/fe-remove-pu-slot
May 17, 2024
0b2f901
Merge remote-tracking branch 'origin/main' into peng/remove-pu-sche
May 20, 2024
ba9e570
Add WorkerId type alias as u32 in mapping.rs
May 20, 2024
4795bb1
Merge branch 'peng/fe-remove-pu-slot' into peng/remove-pu-sche-slot
May 20, 2024
f24ea98
Refactor WorkerSlotMapping and update stream.rs usage
May 20, 2024
915d4fa
Uncomment/implement `to_worker_slot`, add `actor_location` hashmap, r…
May 20, 2024
b85ec7a
Merge remote-tracking branch 'origin/main' into peng/fe-remove-pu-slot
May 20, 2024
2b31097
Merge branch 'peng/fe-remove-pu-slot' into peng/remove-pu-sche-slot
May 20, 2024
7faefb0
Optimize imports in stream_manager.rs
May 20, 2024
395818b
Refactor actor loc tracking and Distribution init
May 20, 2024
839b5b2
Merge remote-tracking branch 'origin/main' into peng/remove-pu-sche-slot
May 21, 2024
56eba87
Merge remote-tracking branch 'origin/main' into peng/fe-remove-pu-slot
May 21, 2024
dd673d8
Merge branch 'peng/fe-remove-pu-slot' into peng/remove-pu-sche-slot
May 21, 2024
366020b
Merge remote-tracking branch 'origin/main' into peng/fe-remove-pu-slot
May 27, 2024
fbdade2
Merge branch 'peng/fe-remove-pu-slot' into peng/remove-pu-sche-slot
May 27, 2024
b79c38b
Merge remote-tracking branch 'origin/main' into peng/fe-remove-pu-slot
May 27, 2024
22edf32
Refactor `catalog.rs` imports, drop `visit_stream_node_cont`
May 27, 2024
6eda8c6
Refactor: Simplify `visit_stream_git_node_cont_mut` import in `catalo…
May 27, 2024
0a7ee88
Merge branch 'peng/fe-remove-pu-slot' into peng/remove-pu-sche-slot
May 27, 2024
1389cae
Merge remote-tracking branch 'origin/main' into peng/remove-pu-sche-slot
May 28, 2024
d2a325d
Fix whitespace
May 28, 2024
4981c0b
Refactor: Consolidate WorkerSlotMapping, rm dupes in mapping.rs
May 28, 2024
3f0764e
Merge remote-tracking branch 'origin/main' into peng/remove-pu-sche-slot
May 31, 2024
6060041
Merge remote-tracking branch 'origin/main' into peng/remove-pu-sche-slot
Jun 3, 2024
a980279
Update Protobuf, Rust vnode and worker mappings
Jun 3, 2024
122099d
Update meta, manager, rpc, stream modules; fix tests and proto
Jun 4, 2024
2d7080d
Merge remote-tracking branch 'origin/main' into peng/remove-pu-sche-slot
Jun 4, 2024
a725cd0
Merge remote-tracking branch 'origin/main' into peng/remove-pu-sche-slot
Jun 17, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Merge remote-tracking branch 'origin/main' into peng/fe-remove-pu-slot
  • Loading branch information
Shanicky Chen committed May 20, 2024
commit b85ec7a370356d70eb6492d4c986c2b81ec3e8af

This merge commit was added into this branch cleanly.

There are no new changes to show, but you can still view the diff.