You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
When user try to create a big MView, it will keep generate a lot of data to RisingWave.
If our system does not compact these data to bottom level, it may cause a back-pressure and let create job stop.
Describe the solution you'd like
There are many problems waiting us to solve:
Meta node does not split state-table which belongs to a creating MV and it means that meta-node would not partition this state-table and it makes our compaction efficiency inefficient.
High throughput write will also cause a large overlapping level and compactor must read a lot of files concurrently for a large overlapping level.
A large task to base level may write too many small files and some object storage may not supply a high performance for small object writing.
According to write-amplification calculate algorithm, we shall pick a compact task to base level as large as possible to reduce write-amplification when there are so much data waiting in level 0 to be compacted.
Describe alternatives you've considered
No response
Additional context
No response
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
When user try to create a big MView, it will keep generate a lot of data to RisingWave.
If our system does not compact these data to bottom level, it may cause a back-pressure and let create job stop.
Describe the solution you'd like
There are many problems waiting us to solve:
Describe alternatives you've considered
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: