Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nix: use -ld_new on macOS #3880

Merged
merged 2 commits into from
Jun 30, 2024
Merged

nix: use -ld_new on macOS #3880

merged 2 commits into from
Jun 30, 2024

Conversation

thoughtpolice
Copy link
Member

The new parallel macOS linker reduces link time for the debug jj binary from 3s to 0.7s on my M2 Macbook Air, which is a significant reduction for nearly no cost at all.

However, enabling it is rather strange because we have to give a full path to the /Applications directory, and I don't know of a good way to encapsulate it with existing Nix primitives inside Nixpkgs...

This change requires Sonoma and Xcode 15, but in theory I think we could target a lower macOS SDK version in order to produce binaries that are more backwards compatible, so the only real cost is that developers who also use Nix would require Sonoma.

Checklist

If applicable:

  • I have updated CHANGELOG.md
  • I have updated the documentation (README.md, docs/, demos/)
  • I have updated the config schema (cli/src/config-schema.json)
  • I have added tests to cover my changes

@thoughtpolice
Copy link
Member Author

This change requires Sonoma and Xcode 15, but in theory I think we could target a lower macOS SDK version in order to produce binaries that are more backwards compatible, so the only real cost is that developers who also use Nix would require Sonoma.

Actually this... doesn't really matter because it's purely for the devShell. So maybe this fix can go ahead and be merged, but I'll think about it some more.

@thoughtpolice thoughtpolice force-pushed the aseipp/push-ovozoptrkkyz branch 3 times, most recently from 8491644 to f169a8d Compare June 20, 2024 00:45
@emilazy
Copy link
Contributor

emilazy commented Jun 24, 2024

This would make me happy. Does it speed up the build for tests meaningfully too?

@thoughtpolice
Copy link
Member Author

This would make me happy. Does it speed up the build for tests meaningfully too?

Probably, but I haven't looked at that exactly. That said this is a small and easy change to make so if someone else likes it, I think it can be merged and backed out if it turns out bad.

@thoughtpolice thoughtpolice marked this pull request as ready for review June 24, 2024 19:35
@emilazy
Copy link
Contributor

emilazy commented Jun 24, 2024

I think you can just use xcrun ld …, by the way. That should work if the user has another developer directory. Or /usr/bin/ld, even.

@emilazy
Copy link
Contributor

emilazy commented Jun 24, 2024

Oh, I guess you can’t use zcrun because Rust is calling it. /usr/bin/ld should work though; those are stubs that redirect to the currently‐active developer directory.

@thoughtpolice thoughtpolice force-pushed the aseipp/push-ovozoptrkkyz branch 2 times, most recently from 56a1e3d to 70abe92 Compare June 29, 2024 02:39
@thoughtpolice
Copy link
Member Author

/usr/bin/ld should work though; those are stubs that redirect to the currently‐active developer directory.

Sure does work, please take a look. /cc'ing @necauqua for review too

@thoughtpolice thoughtpolice requested a review from necauqua June 29, 2024 02:39
Copy link
Contributor

@emilazy emilazy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems to work for me! Just a couple nits…

flake.nix Outdated Show resolved Hide resolved
flake.nix Outdated Show resolved Hide resolved
@emilazy
Copy link
Contributor

emilazy commented Jun 29, 2024

By the way, do we know if this breaks pre‐Sonoma or not? I wonder if maybe the older ld(1) just ignores the argument? If not, it’d be nice if we could make this conditional on it working.

@thoughtpolice
Copy link
Member Author

By the way, do we know if this breaks pre‐Sonoma or not? I wonder if maybe the older ld(1) just ignores the argument? If not, it’d be nice if we could make this conditional on it working.

It probably does, but it only applies to nix develop shells which is probably where it's most useful. Normal nix install routes will continue to use the stdenv linker instead and so will anyone using rustup. So, I think we could conditionalize it to make it safer, but we can probably just wait until someone asks for it or reports a build failure.

@emilazy
Copy link
Contributor

emilazy commented Jun 29, 2024

Breaking support for everything but the latest macOS release makes me a little nervous, but I guess there’s probably, like, five of us actually using the flake to do Jujutsu development, so… sure, why not wait and see :)

flake.nix Outdated Show resolved Hide resolved
@thoughtpolice thoughtpolice force-pushed the aseipp/push-ovozoptrkkyz branch 3 times, most recently from 26ab151 to 0218d93 Compare June 29, 2024 23:21
@necauqua
Copy link
Contributor

I've never heard of running nix on Windows, but wouldn't it be better to just have an empty list there in place of a throw? Just so that in the rare case both checks are false, it wouldn't outright fail because we told it to

Other than that, I don't have a Mac, so I can't test, but it looks ok

@emilazy
Copy link
Contributor

emilazy commented Jun 30, 2024

You kind of can’t run Nix on Windows at present. The most viable third OS is, I think, FreeBSD? I know people are actively working on support for that but I don’t think there’s much upstream at present and I’d be surprised if they have enough working for the Jujutsu flake to work. Agreed that an empty list is preferable though.

The new parallel macOS linker reduces link time for the debug `jj` binary from
3s to 0.7s on my M2 Macbook Air, which is a significant reduction for nearly
no cost at all. This only assumes that you have a new enough Xcode environment
as your default (where `/usr/bin/ld` resides.)

This change requires Sonoma and Xcode 15, but in theory I think we could target
a lower macOS SDK version in order to produce binaries that are more backwards
compatible, so the only real cost is that developers who also use Nix would
require Sonoma.
The relevant upstream issue has been fixed since version 0.9.72,
as noticed by Emily.

Signed-off-by: Austin Seipp <[email protected]>
@thoughtpolice thoughtpolice force-pushed the aseipp/push-ovozoptrkkyz branch from 0218d93 to 14ed4dc Compare June 30, 2024 19:24
@thoughtpolice
Copy link
Member Author

I reworked it to use an empty list. Thanks!

@thoughtpolice thoughtpolice merged commit a3ca701 into main Jun 30, 2024
29 checks passed
@thoughtpolice thoughtpolice deleted the aseipp/push-ovozoptrkkyz branch June 30, 2024 19:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants