feat(config): make arrangement backfill default #14846
+1,071
−1,359
Merged
Task list completed / task-list-completed
Started
2024-05-27 04:44:49
ago
0 / 8 tasks completed
8 tasks still to be completed
Details
Required Tasks
Task | Status |
---|---|
end-to-end test 19min -> 21min | Incomplete |
end-to-end test parallel 10min -> 12min | Incomplete |
end-to-end test deterministic sim 21min -> 25min (Could be just regression from other pipelines..) | Incomplete |
Backfill test seem to improve. | Incomplete |
e2e release test seem to improve. | Incomplete |
Instead of just using storage table, we now use ReplicatedStateTable . Metadata is larger since we need the full TableCatalog , instead of just TableDesc . |
Incomplete |
On recovery, we need to read backfill state and recover the vnode level state. | Incomplete |
#14846 (comment) | Incomplete |
I have written necessary rustdoc comments | Incomplete |
I have added necessary unit tests and integration tests | Incomplete |
I have added test labels as necessary. See details. | Incomplete |
I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features #7934). | Incomplete |
My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future). | Incomplete |
All checks passed in ./risedev check (or alias, ./risedev c ) |
Incomplete |
My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details) | Incomplete |
My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users) | Incomplete |
Why e2e test will TLE. (Perhaps it's the get_row each time we persist state in debug mode). |
Incomplete |
Why deterministic test test_high_barrier_latency_cancel takes a long time. |
Incomplete |
Why recovery test fails. | Incomplete |
https://buildkite.com/risingwave-test/backfill/builds/387 | Incomplete |
https://buildkite.com/risingwave-test/backfill/builds/386 | Incomplete |
no shuffle w tomb: https://buildkite.com/risingwave-test/backfill/builds/380#018e2f1b-c128-47ef-9cf5-1e5520587b05 | Incomplete |
no shuffle w/o tomb: https://buildkite.com/risingwave-test/backfill/builds/379#018e2ee5-adf4-46a1-8fb9-b6a92e0c2a60 | Incomplete |
arrangement w tomb https://buildkite.com/risingwave-test/backfill/builds/387 | Incomplete |
arrangement no tomb https://buildkite.com/risingwave-test/backfill/builds/386 | Incomplete |
set streaming_use_arrangement_backfill back to false by default. |
Incomplete |
Run ./risedev dapt to reset stream scan to use backfill rather than arrangement backfill. |
Incomplete |
Backfill to NoShuffleBackfill |
Incomplete |
ArrangementBackfill to Backfill |
Incomplete |
Loading