Enhance(compaction): Improving the efficiency of the trivial-move task

[Li0k]

https://grafana.test.risingwave-cloud.xyz/d/EpkBw5W4k/risingwave-dev-dashboard?from=1732025046079&orgId=1&to=1732031886092&var-component=All&var-datasource=cdtasocg64074c&var-instance=risingwave&var-namespace=rwc-g1id2bbsgif4fou057af8nafim-li0k-test2&var-pod=All&var-table=All

Recently, in my huge ckpt size test, I found that append only table triggers a huge number of trivial-move tasks.

However, hummock currently only supports single sst trivial-move task. Even though, for trivial-move task we did some commit optimization (merge up to 256 trivial-move task commits), it may still trigger a bottleneck in the meta for large number of ssts.

Therefore, we can support multi sst trival-move tasks to optimize this performance issue.

There are two Pickers in Hummock that can generate trival-move tasks

• TrivialMovePicker
• MinOverlappingPicker

1. consider extending the single-file algorithm to multi-file, but this may be more computationally intensive.
2. Allow batch commit parameters to be changed at runtime to improve commit efficiency.