Tracking issue: remove and mitigate __attrsFailEvaluation

[philiptaron]

This is all related to the following PR:

• Ensure all jobset attributes evaluate, and check that in CT #269356

__attrsFailEvaluation

From pkgs/top-level/release-attrpaths-superset.nix, written by @amjoseph-nixpkgs (now deleted).

__attrsFailEvaluation is a temporary workaround to get top-level eval to succeed (under builtins.tryEval) for the entire packageset, without deep invasive changes into individual packages.

Now that CI has been added, ensuring that top-level eval will not be broken by any new commits, you should not add any new occurrences of __attrsFailEvaluation, and should remove them wherever you are able to (doing so will likely require deep adjustments within packages). Once all of the uses of __attrsFailEvaluation are removed, it will be deleted from the routine below.

In the meantime, the intended semantics are that an attrpath rooted at pkgs is part of the (unfiltered) release jobset if and only if all of the following are true:

1. The first component of the attrpath is not in excluded-toplevel-attrs.
2. No attrname in the attrpath belongs to the list of forbidden attrnames excluded-attrnames-at-any-depth.
3. The attrpath leads to a value for which lib.isDerivation is true.
4. No proper prefix of the attrpath has __attrsFailEvaluation = true;.
5. Any proper prefix of the attrpath at which lib.isDerivation is true also has __recurseIntoDerivationForReleaseJobs = true;.

The last condition is unfortunately necessary because there are Hydra release job names which have proper prefixes which are attrnames of derivations (!). We should probably restructure the job tree so that this is not the case.

• nixpkgs-lib test pkgs/top-level/release-attrpaths-superset.nix infinite recurses on eval-system=aarch64-darwin #319147
• Fix failures with pkgs/top-level/release-attrpaths-superset.nix #319220
• lib.types: remove __attrsFailEvaluation #323854
• nodePackages: remove __attrsFailEvaluation in all-packages #323857
• dockerTools: remove __attrsFailEvaluation #323860
• eclipses.plugins: remove __attrsFailEvaluation #323865
• pidginPackages: move __attrsFailEvaluation to allow deeper evaluation #323867
• emacsPackages: remove __attrsFailEvaluation #323868
• qt5: remove __attrsFailEvaluation as all attrsets evaluate #324622
• rustPackages.buildRustPackages: allow one level of introspection before applying __attrsFailEvaluation #324633
• coqPackages: allow one level of introspection before applying __attrsFailEvaluation #324675
• javaPackages.compiler.adoptopenjdk: all attrsets evaluate, so remove __attrsFailEvaluation #324680
• pythonPackages: move __attrsFailEvaluation to allow deeper inspection #324682
• cudaPackages: move __attrsFailEvaluation to allow deeper inspection #324685
• linuxPackages: remove __attrsFailEvaluation since the attrset evaluates #324688
• haskellPackages: remove __attrsFailEvaluation, buildHaskellPackages, and generateOptparseApplicativeCompletions special cases #324691

More to come, maybe.

excluded-toplevel-attrs

From pkgs/top-level/release-attrpaths-superset.nix, written by @amjoseph-nixpkgs (now deleted).

No release package attrpath may have any of these attrnames as its initial component.

If you can find a way to remove any of these entries without causing CI to fail, please do so.

• __splicedPackages
• pkgsBuildBuild
• pkgsBuildHost
• pkgsBuildTarget
• pkgsHostHost
• pkgsHostTarget
• pkgsTargetTarget
• buildPackages
• targetPackages
• pkgsLLVM
• pkgsMusl
• pkgsStatic
• pkgsCross
• pkgsx86_64Darwin
• pkgsi686Linux
• pkgsLinux
• pkgsExtraHardening

excluded-attrnames-at-any-depth

From pkgs/top-level/release-attrpaths-superset.nix, written by @amjoseph-nixpkgs (now deleted).

No release package attrname may have any of these at a component anywhere in its attrpath. These are the names of gigantic top-level attrsets that have leaked into so many sub-packagesets that it's easier to simply exclude them entirely.

If you can find a way to remove any of these entries without causing CI to fail, please do so.

• lib
• override
• __functor
• __functionArgs
• __splicedPackages
• newScope
• scope
• pkgs
• test-pkgs Remove test-pkgs, buildPackages, and buildFreebsd from excluded-attrnames-at-any-depth #324696
• buildHaskellPackages haskellPackages: remove __attrsFailEvaluation, buildHaskellPackages, and generateOptparseApplicativeCompletions special cases #324691
• buildPackages Remove test-pkgs, buildPackages, and buildFreebsd from excluded-attrnames-at-any-depth #324696
• buildFreebsd Remove test-pkgs, buildPackages, and buildFreebsd from excluded-attrnames-at-any-depth #324696
• generateOptparseApplicativeCompletions haskellPackages: remove __attrsFailEvaluation, buildHaskellPackages, and generateOptparseApplicativeCompletions special cases #324691
• callPackage
• mkDerivation
• overrideDerivation
• overrideScope
• overrideScope'

Special case: lib/types.nix leaks into a lot of NixOS-related derivations, and does not eval deeply.

• type

__recurseIntoDerivationForReleaseJobs

nixpkgs/pkgs/test/default.nix

Lines 132 to 136 in a1ecdd6

	php = recurseIntoAttrs (callPackages ./php {});
	pkg-config = recurseIntoAttrs (callPackage ../top-level/pkg-config/tests.nix { }) // { __recurseIntoDerivationForReleaseJobs = true; };
	buildRustCrate = callPackage ../build-support/rust/build-rust-crate/test { };

https://github.com/NixOS/nixpkgs/blob/a1ecdd6b1ad1dcf345ca0a83e9a96b391e951544/pkgs/top-level/all-packages.nix#L15766

https://github.com/NixOS/nixpkgs/blob/a1ecdd6b1ad1dcf345ca0a83e9a96b391e951544/pkgs/top-level/all-packages.nix#L23787

https://github.com/NixOS/nixpkgs/blob/a1ecdd6b1ad1dcf345ca0a83e9a96b391e951544/pkgs/top-level/all-packages.nix#L30260

nixpkgs/pkgs/top-level/coq-packages.nix

Line 223 in a1ecdd6

in cp // { coqPackages = cp.coqPackages // { __attrsFailEvaluation = true; }; } // { __recurseIntoDerivationForReleaseJobs = true; };

nixpkgs/pkgs/top-level/linux-kernels.nix

Line 680 in a1ecdd6

__recurseIntoDerivationForReleaseJobs = true;

nixpkgs/pkgs/top-level/pkg-config/tests.nix

Line 20 in a1ecdd6

defaultPkgConfigPackages = allPkgs.callPackage ./test-defaultPkgConfigPackages.nix { } // { __recurseIntoDerivationForReleaseJobs = true; };

nixpkgs/pkgs/top-level/release-python.nix

Line 31 in a1ecdd6

else if value.recurseForDerivations or false || value.recurseForRelease or false || value.__recurseIntoDerivationForReleaseJobs or false then