Compute node OOM for nexmark queries with nightly-20240108

[cyliu0]

Describe the bug

We have a lot of OOM for the daily nexmark performance test pipelines with nightly-20240108.
https://buildkite.com/risingwave-test/nexmark-benchmark/builds/2827
https://buildkite.com/risingwave-test/nexmark-benchmark/builds/2828
https://buildkite.com/risingwave-test/nexmark-benchmark/builds/2829

The memory usage are all like this build https://buildkite.com/risingwave-test/nexmark-benchmark/builds/2827

Error message/log

No response

To Reproduce

No response

Expected behavior

No response

How did you deploy RisingWave?

No response

The version of RisingWave

nightly-20240108

Additional context

The new commits in nightly-20240108 https://github.com/risingwavelabs/rw-commits-history?tab=readme-ov-file#nightly-20240108

[huangjw806]

The same OOM issue occurred longevity test pipeline.
https://buildkite.com/risingwave-test/longevity-test/builds/894
https://buildkite.com/risingwave-test/longevity-test/builds/895

[xxchan]

This can be caused by #14375 (comment) (This is resolved by #14427) + #14220

1704728569-2024-01-08-15-42-48.auto.heap.collapsed.zip

[huangjw806]

#14375 cherry-picked into release-1.6, should we cherry-pick #14427 into release-1.6?

[cyliu0]

No OOM for nightly-20240109 daily performance test. So close this one.