You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, RisingWave batch queries can easily consume too much memory without limitation. This is a great threat to our system's stability. To resolve this issue, we can first implement memory control logic to ensure that if batch queries consume more memory than the reserved memory limit, we can abort a batch query quickly to avoid a potential OOM. The memory check could be implemented on operators that could consume a large amount of memory, e.g. HashAgg, Hash Join, and so on. After we have a memory control logic which means now we know exactly the moment of not enough memory, we can implement a spill-to-disk algorithm for those batch operators.
Currently, RisingWave batch queries can easily consume too much memory without limitation. This is a great threat to our system's stability. To resolve this issue, we can first implement memory control logic to ensure that if batch queries consume more memory than the reserved memory limit, we can abort a batch query quickly to avoid a potential OOM. The memory check could be implemented on operators that could consume a large amount of memory, e.g. HashAgg, Hash Join, and so on. After we have a memory control logic which means now we know exactly the moment of not enough memory, we can implement a spill-to-disk algorithm for those batch operators.
The text was updated successfully, but these errors were encountered: