You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Previously, we introduced a SnapshotManager to pin/unpin a shapshot when creating materialize view, which fix #5592. And later, with the implementation of #6341, chain/backfill executor will execute NoWait read, and seems that we don't need to pin shapshot anymore.
However, there is an issues: current implementation require strong binding between mv and chain. If we decouple chain and mv, chain executor need to wait epoch, but it may cause live lock.
If barrier 1 is creating mv, chain need to wait epoch 1. But if chain can wait epoch 1, epoch 1 must have flowed through the global. So actually chain buffer all the data behind epoch 1, including upstream change. And this is conflict with concurrent checkpoint.
I'm not sure whether will we decouple chain and materialize executor, so I'd like to draft a PR to avoid pin snapshot, and to ensure correctness.
It's ok to remove snapshot manager in barrier now.
Currently, we will pin a hummock snapshot on mv creation and unpin it when mv creation completes.
IIUC, after #6341, we no longger need to pin hummock snapshot since the backfill executor always scan the latest snapshot
The text was updated successfully, but these errors were encountered: