Tracking: optimize performance of hummock manager in meta-node

[Little-Wallace]

Background

Some user meet a problem which is the meta-node use too much CPU resource (nearly 100%, one whole core), and hummock manager can not generate compact task in time.

Develop Task

• optimize metrics report bug(meta): Hummock's operations consume a lot of meta cpu #15153
• reduce object clone of HummockVersion (feat(storage): replace hummock protobuf strcut with rust struct #15386)
• optimize ETCD (SQL-Backend) transaction cost , batch several compact task for once ETCD. Maybe SQL-Backend could solve part of this problem. But in current implementation, all operations of HummockManager would acquire the same lock and update the same key in ETCD, So it is still important to batch operations.

[github-actions]

This issue has been open for 60 days with no activity. Could you please update the status? Feel free to continue discussion or close as not planned.

[github-actions]

This issue has been open for 60 days with no activity.

If you think it is still relevant today, and needs to be done in the near future, you can comment to update the status, or just manually remove the no-issue-activity label.

You can also confidently close this issue as not planned to keep our backlog clean.
Don't worry if you think the issue is still valuable to continue in the future.
It's searchable and can be reopened when it's time. 😄