Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cc-wrapper: clean-up post stdenvAdapters.useLibsFrom: init #281750

Closed

Conversation

SomeoneSerge
Copy link
Contributor

...from "cudaPackages.backendStdenv: useGccForLibs", 210ce38

Description of changes

#275947 (based on @rrbutani's suggestions and PoC) introduced (without approval) passthru attributes in the cc-wrapper, the purpose of which is to allow propagating the standard c++ libraries used by another wrapper. These attributes are now (ab)used to fix the libstdc++ compatibility issues in cudaSupport = true packages and in python3Packages.tensorflow, which use older gcc than the rest of nixpkgs.

As #275947 (comment) points out, this had introduced more junk in the wrapper, lacks longer-term vision, etc.

This draft is a placeholder just removing the not-so-useful comments introduced by the preceding PR, but I'm hoping we can also have a conversation about whether the passthru solution hits a local minima, about what the other cc-wrapper bits touched by the PR were actually intended for, etc.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

...from "cudaPackages.backendStdenv: useGccForLibs",
210ce38
@SomeoneSerge SomeoneSerge requested a review from rrbutani January 18, 2024 10:34
Copy link
Contributor Author

@SomeoneSerge SomeoneSerge Jan 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This:

# Analogously to cc_solib and gccForLibs_solib
libcxx_solib = "${lib.getLib libcxx}/lib";

used to be this:

echo " -L${lib.getLib libcxx}/lib" >> $out/nix-support/cc-ldflags

Afaiu this was exactly the same logic as

+ optionalString useGccForLibs ''
echo "-L${gccForLibs}/lib/gcc/${targetPlatform.config}/${gccForLibs.version}" >> $out/nix-support/cc-ldflags
echo "-L${gccForLibs_solib}/lib" >> $out/nix-support/cc-ldflags

...is that correct?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As the description mentions, the prime motivation of the cc-wrapper change was to be able to inherit the nixpkgs' default stdenv's libraries (primarily, libstdc++ on {x86_64,aarch64}-linux) in compat stdenvs, e.g. when using an older gcc.

I still find it questionable e.g. to

  1. pass wrappCCWith the wrapped (g)cc instead of of the unwrapped lib output,
  2. pass the stdenv instead of the lib in stdenvAdapters.useLibsFrom.

@SomeoneSerge SomeoneSerge changed the title cc-wrapper: drop unhelpful TODO comments from... cc-wrapper: clean-up post `s Jan 18, 2024
@SomeoneSerge SomeoneSerge changed the title cc-wrapper: clean-up post `s cc-wrapper: clean-up post stdenvAdapters.useLibsFrom: init Jan 18, 2024
@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux labels Jan 18, 2024
Copy link
Contributor Author

@SomeoneSerge SomeoneSerge Jan 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In relation to #281371: it isn't enough to propagate the pkgs.stdenv's libraries to the adapted stdenv, it's also necessary to take the other stdenv's (e.g. pkgs.gcc11Stdenv's) native platform's tools, e.g. coreutils:

# The wrapper scripts use 'cat' and 'grep', so we may need coreutils.
coreutils_bin = optionalString (!nativeTools) (getBin coreutils);

Currently these aren't exposed in passthru or anyhow (except for env, which barely can be considered a part of the public interface)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The question regarding

/* NOTE: cc.cc is the unwrapped compiler. Should we respect the old
* wrapper instead? */
cc = targetStdenv.cc.cc;
is "do cc-wrappers compose"? In the current form, useLibsFrom pkgs.stdenv pkgsStatic.gcc11Stdenv is going to discard all of the flags related to static linkage I'm pretty sure

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant