RFC: cargo: add vendored-libgit2 feature

[ilyagr]

It is only active if enabled manually at this point.

It seems to work insofar as otool -L stops listing libgit2 as a dynamic dependency when --feature vendored-libgit2 is used.

I'm not 100% sure, but perhaps we could recommend that Homebrew uses --feature vendored-libgit2 and maybe --feature-vendoredopenssl. We could also use it in our builds.

[ilyagr]

I'm also not sure how our musl, supposedly static, builds work without this feature.

Update: There's something I'm missing. The official Mac aarch64 does not dynamically link to libgit2 according to otool -L. I should look up how that's built.

Update 2: According to @bnjmnt4n on Discord, what is happening is that git2 automatically statically links libgit2 if it cannot find a libgit2 to dynamically link to. So, since CI doesn't have libgit2 installed, it gets statically linked. But if jj is compiled on a computer with Homebrew & libgit2 of the correct version installed, it's dynamically linked, and then jj will stop working once libgit2 is upgraded.

[emilazy]

I don’t know what Homebrew’s policy on vendoring is, but speaking as a Nixpkgs maintainer, generally package distributions don’t like to use vendored library code. It makes tracking source provenance harder, causes duplicated work when there are build issues in a library that require downstream patching, and can bite you hard when a library gets a security fix. Some are stricter than others, though, so maybe Homebrew would be fine with this.

[thoughtpolice]

Related: #2896 in which the 0.13 binary release got dynamically linked against libgit2, somehow. I wonder if we got a reused runner that somehow had libgit2 installed via brew previously, somehow...

[thoughtpolice]

I don’t know what Homebrew’s policy on vendoring is, but speaking as a Nixpkgs maintainer, generally package distributions don’t like to use vendored library code. It makes tracking source provenance harder, causes duplicated work when there are build issues in a library that require downstream patching, and can bite you hard when a library gets a security fix. Some are stricter than others, though, so maybe Homebrew would be fine with this.

I think there's a strong case to be made that libgit2 is so integral to us from a feature and compatibility POV that it should be vendored inside every build on every supported platform, actually, and that Homebrew and Nix currently behave incorrectly (they both have requirements on external dependencies). There is no other version of libgit2 we test or support in any capacity, its behavior is absolutely critical and essential to every user facing function that currently exists, and behavioral changes may still surface between versions even if APIs don't, like the recent 0.19.0 git2 update has shown (#4080).

Hell, we can't even be sure that upstreams run all the tests before they push out a new version, or test the upgrade process for existing users carefully. It sucks but that's what it is. There is absolutely no reality that exists in which a package maintainer is in a better position to make this call than us.

It's all a bit of a joke, anyway, if you ask me. The moment we start using Gix fully and throw out libgit2, suddenly, the entire requirement to unvendor stuff and share it through a stable ABI will magically go away (along with the implicit requirement to then support the 50 different build variations distros can think up). Because most upstreams have decided that unvendoring Cargo/Rust code, which is (and I'm painting a very broad stroke here) explicitly against Cargo's design and the broader Rust cultural mores, is more work than just duplicating the effort or fixing packages directly. So, in that sense, the problem will eventually solve itself at least. :)

[emilazy]

You actually totally can unvendor Rust dependencies with a custom Cargo repository and Nixpkgs is considering doing it :)

But I’m not trying to make a value statement here, just conveying the reality of what many downstream distributions are likely to think about it. Especially Fedora.

I’d love to see us move over to Gitoxide anyway and have worked on it locally (but pushes are still a big problem).

[ilyagr]

I think I'll close this PR in favor of #4163. This is similar to what @thoughtpolice proposed: it enables the feature for good. However, it turns out that this is probably what we want even if somebody wants to dynamically link libgit2, as an environment variable is a better way to do that (as opposed to disabling the feature).