Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(storage): decouple upload task finish order from epoch #17460

Merged
merged 30 commits into from
Jul 9, 2024
Merged
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
4fa032a
refactor(storage): remove sync finish event
wenym1 Jun 18, 2024
09950fc
syncing data upload tasks partial succeed
wenym1 Jun 18, 2024
85b2ded
extract poll_syncing_task
wenym1 Jun 18, 2024
892dd29
remove uploader event
wenym1 Jun 18, 2024
2874313
fix compile
wenym1 Jun 19, 2024
7bd9c79
Merge branch 'yiming/remove-sync-finish-event' into yiming/dag-uploader
wenym1 Jun 24, 2024
82a4191
feat(storage): decouple upload task finish order from epoch
wenym1 Jun 25, 2024
f2fa4de
Merge branch 'main' into yiming/remove-sync-finish-event
wenym1 Jun 26, 2024
af22875
Merge branch 'yiming/remove-sync-finish-event' into yiming/dag-uploader
wenym1 Jun 26, 2024
eaac3f9
move to new file
wenym1 Jun 26, 2024
00e9d00
Merge branch 'main' into yiming/remove-sync-finish-event
wenym1 Jun 26, 2024
b0aad6d
Merge branch 'yiming/remove-sync-finish-event' into yiming/dag-uploader
wenym1 Jun 26, 2024
372dfc4
Merge branch 'main' into yiming/dag-uploader
wenym1 Jun 26, 2024
4136e88
track spill table id
wenym1 Jun 27, 2024
5aef226
add check and refactor
wenym1 Jun 28, 2024
e51e186
temp save
wenym1 Jun 28, 2024
d234bbe
refactor(storage): remove global epoch in uploader
wenym1 Jun 28, 2024
37cc33d
Merge branch 'yiming/uploader-no-global-epoch' into yiming/dag-uploader
wenym1 Jun 28, 2024
30a808e
fix test and address comment
wenym1 Jul 2, 2024
8d6badd
Merge branch 'main' into yiming/uploader-no-global-epoch
wenym1 Jul 8, 2024
ffb9906
fix panic
wenym1 Jul 8, 2024
3808d55
fix
wenym1 Jul 9, 2024
abf2ff8
Merge branch 'yiming/uploader-no-global-epoch' into yiming/dag-uploader
wenym1 Jul 9, 2024
d02775a
print error table id
wenym1 Jul 9, 2024
62b5cd4
ack sync with subset
wenym1 Jul 9, 2024
aa562f8
Merge branch 'yiming/uploader-no-global-epoch' into yiming/dag-uploader
wenym1 Jul 9, 2024
1010dbf
fix clippy
wenym1 Jul 9, 2024
0d30c07
Merge branch 'yiming/uploader-no-global-epoch' into yiming/dag-uploader
wenym1 Jul 9, 2024
1a3f2e7
Merge branch 'main' into yiming/uploader-no-global-epoch
wenym1 Jul 9, 2024
ddab2d3
Merge branch 'yiming/uploader-no-global-epoch' into yiming/dag-uploader
wenym1 Jul 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -469,9 +469,9 @@ impl HummockEventHandler {
table_ids: HashSet<TableId>,
) {
debug!(
"awaiting for epoch to be synced: {}, max_synced_epoch: {}",
new_sync_epoch,
self.uploader.max_synced_epoch()
?table_ids,
"awaiting for epoch to be synced",
);
self.uploader
.start_sync_epoch(new_sync_epoch, sync_result_sender, table_ids);
Expand All @@ -481,7 +481,6 @@ impl HummockEventHandler {
info!(
prev_epoch,
max_committed_epoch = self.uploader.max_committed_epoch(),
max_synced_epoch = self.uploader.max_synced_epoch(),
"handle clear event"
);

Expand Down
Loading
Loading