feat(stream): wait committed epoch in state table init_epoch #19223
Task list completed / task-list-completed
Started
2024-11-12 06:56:36
ago
1 / 8 tasks completed
7 tasks still to be completed
Details
Required Tasks
| Task | Status |
|---|---|
| I have written necessary rustdoc comments | Incomplete |
| I have added necessary unit tests and integration tests | Incomplete |
| I have added test labels as necessary. See details. | Incomplete |
| I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features #7934). | Incomplete |
| My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future). | Incomplete |
All checks passed in ./risedev check (or alias, ./risedev c) |
Completed |
| My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details) | Incomplete |
| My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users) | Incomplete |
In the init of LocalHummockStorage, we change to wait for the prev_epoch to be the committed_epoch no matter whether it is is_replicated or not. Previously we only do this when is_replicated is true. |
Incomplete |
In clear_shared_buffer, we no longer pass the version_id to wait for. In event handler, since we don't need to wait for bumping the version id, the handling logic becomes non-async. As a result, in the InitRequest of streaming control bidi-stream, we won't pass the version_id. |
Incomplete |
| Some unit tests are updated to adapt to the change. | Incomplete |
All stateful executor should strictly follow an order in initialization: receive first barrier -> save some barrier information -> yield barrier -> call StateTable::init_epoch. |
Incomplete |
In this PR, we ensure that all usages should be included in the changed code. Changes in most executors are trivial, simply reordering some codes. now.rs and temporal_join.rs may require extra attention, because the change is relatively not that trivial. |
Incomplete |
The init_epoch of StateTable is changed from non-async to async, and therefore all direct usages on the methods will need to add .await, and therefore included in the changed code. |
Incomplete |
For any non-async methods that previously call StateTable::init_epoch, they become async, and therefore all usages on these methods will need to add .await, and therefore included in the changed code. |
Incomplete |
For async methods that previously call StateTable::init_epoch, they will renamed to init_after_yield_barrier, and included in the changed code. |
Incomplete |
Loading