chore: use system linker in macOS

[BugenZhao]

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

The performance of the system linker has improved a lot in Xcode 15 (shipped with macOS 14 and enabled by default). See https://news.ycombinator.com/item?id=36218330.

A simple benchmark on compiling cmd_all and linking the binary:

ld-new (default) 9.38s
lld             13.40s
ld-classic      14.45s

I believe it could outperforms further more under the release profile.

So let's switch to the system linker to improve the experience and reduce the number of dependencies.

Checklist

• I have written necessary rustdoc comments
• I have added necessary unit tests and integration tests
• All checks passed in ./risedev check (or alias, ./risedev c)

Documentation

• My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note

If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.

[fuyufjh]

LGTM. Still don't know why I ran into that weird issue.

[xxchan]

LGTM. Still don't know why I ran into that weird issue.

We might need to double-check this before merging

[BugenZhao]

Still don't know why I ran into that weird issue.

I've reproduced the same issue. 😕 This appears to be a long-standing issue of linkme not correctly work under macOS if inter-crate lto is not enabled: dtolnay/linkme#61. cc @wangrunji0408 Do you have any ideas?

Some observations:

• We also use linkme for system catalogs. It still works well. The difference is that all definitions of system catalogs live in the same crate (frontend).
• Homebrew build of RisingWave has already switched to the system linker since risingwavelabs/homebrew-risingwave@73103d3. However, there's no such issue. I suppose it's because we enable lto for release build.
• The LLVM linker does not have this issue either. Not sure if the problem only applies to Apple's linker.

[wangrunji0408]

If linkme continues to make trouble, we can replace it by https://github.com/dtolnay/inventory. It relies on ctor and doesn't have too much link-time magics.

[BugenZhao]

If linkme continues to make trouble, we can replace it by dtolnay/inventory. It relies on ctor and doesn't have too much link-time magics.

Wow, so much magic! I think either of them works for us. I was once curious why linkme uses a complete different approach instead of simply wrapping ctor (or why don't we simply do this). It turns out to be an intentional design.

[BugenZhao]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• chore: use system linker in macOS #15407 👈
• refactor: replace linkme with inventory #15631
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @BugenZhao and the rest of your teammates on Graphite

[BugenZhao]

If linkme continues to make trouble, we can replace it by dtolnay/inventory. It relies on ctor and doesn't have too much link-time magics.

Wow, so much magic! I think either of them works for us. I was once curious why linkme uses a complete different approach instead of simply wrapping ctor (or why don't we simply do this). It turns out to be an intentional design.

#15631 (comment) 🥲