Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(config): make arrangement backfill default #14846

Merged
merged 19 commits into from
Mar 27, 2024

bump timeout again

bbf0dd6
Select commit
Loading
Failed to load commit list.
Merged

feat(config): make arrangement backfill default #14846

bump timeout again
bbf0dd6
Select commit
Loading
Failed to load commit list.
Task list completed / task-list-completed Started 2024-05-27 04:44:49 ago

0 / 8 tasks completed

8 tasks still to be completed

Details

Required Tasks

Task Status
end-to-end test 19min -> 21min Incomplete
end-to-end test parallel 10min -> 12min Incomplete
end-to-end test deterministic sim 21min -> 25min (Could be just regression from other pipelines..) Incomplete
Backfill test seem to improve. Incomplete
e2e release test seem to improve. Incomplete
Instead of just using storage table, we now use ReplicatedStateTable. Metadata is larger since we need the full TableCatalog, instead of just TableDesc. Incomplete
On recovery, we need to read backfill state and recover the vnode level state. Incomplete
#14846 (comment) Incomplete
I have written necessary rustdoc comments Incomplete
I have added necessary unit tests and integration tests Incomplete
I have added test labels as necessary. See details. Incomplete
I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features #7934). Incomplete
My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future). Incomplete
All checks passed in ./risedev check (or alias, ./risedev c) Incomplete
My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details) Incomplete
My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users) Incomplete
Why e2e test will TLE. (Perhaps it's the get_row each time we persist state in debug mode). Incomplete
Why deterministic test test_high_barrier_latency_cancel takes a long time. Incomplete
Why recovery test fails. Incomplete
https://buildkite.com/risingwave-test/backfill/builds/387 Incomplete
https://buildkite.com/risingwave-test/backfill/builds/386 Incomplete
no shuffle w tomb: https://buildkite.com/risingwave-test/backfill/builds/380#018e2f1b-c128-47ef-9cf5-1e5520587b05 Incomplete
no shuffle w/o tomb: https://buildkite.com/risingwave-test/backfill/builds/379#018e2ee5-adf4-46a1-8fb9-b6a92e0c2a60 Incomplete
arrangement w tomb https://buildkite.com/risingwave-test/backfill/builds/387 Incomplete
arrangement no tomb https://buildkite.com/risingwave-test/backfill/builds/386 Incomplete
set streaming_use_arrangement_backfill back to false by default. Incomplete
Run ./risedev dapt to reset stream scan to use backfill rather than arrangement backfill. Incomplete
Backfill to NoShuffleBackfill Incomplete
ArrangementBackfill to Backfill Incomplete