refactor(meta): commit finish catalog in barrier manager #17428
Merged
Task list completed / task-list-completed
Started
2024-07-16 09:39:13
ago
0 / 8 tasks completed
8 tasks still to be completed
Details
Required Tasks
| Task | Status |
|---|---|
In order for the barrier manager to be able to finish stream job, for v1 we need to pass the entire StreamingJob struct currently. |
Incomplete |
This is not necessary for v2, where we only need the streaming job id, and table to replace (if any). You may look at TrackingJob::pre_finish for these logics. |
Incomplete |
As such perhaps we should separate these parameters being passed to the Command::CreateStreamingJob, to avoid redundant cloning / moving of the StreamingJob struct. |
Incomplete |
| I have written necessary rustdoc comments | Incomplete |
| I have added necessary unit tests and integration tests | Incomplete |
| I have added test labels as necessary. See details. | Incomplete |
| I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features #7934). | Incomplete |
| My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future). | Incomplete |
All checks passed in ./risedev check (or alias, ./risedev c) |
Incomplete |
| My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details) | Incomplete |
| My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users) | Incomplete |
| when register a tx on a table id, acquire lock and check the job status. If job is creating, store the tx. If the job has been created, return with ok. If the job does not exist, return err to tell the user about it. | Incomplete |
On abort_all, notify all tx with err. |
Incomplete |
| Every drop/cancel streaming job ddl should remove the tx of the dropped/cancelled streaming job id. | Incomplete |
| job has been created, and then we just return ok, with the global latest notification version | Incomplete |
failed to collect the first barrier. Stream manager should have got an error before calling wait_streaming_job, no need to handle. |
Incomplete |
First barrier collected, and job status must have been turned into Creating. Even though there can be error and recovery happened, we can just store the tx anyway. It will be notified either Ok when the job finishes backfill afterward, or Err when a later error triggers a recovery and call abort_all. |
Incomplete |
Loading