Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python3Packages: Fix and document the removal of the local .overrideAttrs attribute #350127

Conversation

ShamrockLee
Copy link
Contributor

@ShamrockLee ShamrockLee commented Oct 21, 2024

This PR cleans up the leftover of #333670.

After #333670, buildPythonPackage and buildPythonApplication no longer apply lib.makeOverridable themselves and no longer provide .override and .overrideDerivation of the returning package.

For Python packages provided by Nixpkgs, the local .override attribute is almost always shadowed by the one provided by callPackage, and so does the .overrideDerivation attribute. I didn't find any use of the local override in Nixpkgs, but @trofi reported the evaluation failure of python3Packages.pythonCatchConflictsHook.tests.catches-conflict-multiple-chains due to the loss of the local overrideDerivation.

This PR uses lib.overrideDerivation instead of the overrideDerivation attribute to fix the evaluation of python3Packages.pythonCatchConflictsHook.tests.catches-conflict-multiple-chains and document the removal of the buildPython*-generated local overrideDerivation attribute in the release note.

Update: As the Nixpkgs 24.11 branch-off is made, this PR should be backported onto the release-24.11 branch once it is merged.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@github-actions github-actions bot added 6.topic: python 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: documentation This PR adds or changes documentation 8.has: changelog labels Oct 21, 2024
@nix-owners nix-owners bot requested a review from natsukium October 21, 2024 02:47
@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 1-10 labels Oct 21, 2024
@ShamrockLee ShamrockLee marked this pull request as draft October 21, 2024 06:45
@ShamrockLee
Copy link
Contributor Author

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review


x86_64-linux

⏩ 1 package blacklisted:
  • nixos-install-tools

@ShamrockLee ShamrockLee marked this pull request as ready for review October 21, 2024 17:54
@trofi
Copy link
Contributor

trofi commented Oct 21, 2024

The change fixes the eval failures for that attribute for me. Thank you!

But I'm afraid I found another eval failure:

$ nix-instantiate -A python3Packages.scikit-image.tests.all-tests
error: attribute 'override' missing
       at /tmp/z/nixpkgs/pkgs/development/python-modules/scikit-image/default.nix:160:19:
          159|     passthru.tests = {
          160|       all-tests = self.override { doCheck = true; };
             |                   ^
          161|     };

It bisects down to the same 58bfe74 buildPython*: Deprecate and remove (buildPython* { ... }).override. Worth fixing it as well?

…ge { ... }).overrideDerivation

Clean up the leftover of commit 58bfe74 ("buildPython*:
Deprecate and remove (buildPython* { ... }).override")
…Derivation

Use lib.overrideDerivation instead of <pkg>.overrideDerivation
to fix the evaluation of
python3Packages.pythonCatchConflictsHook.tests.catches-conflict-multiple-chains,
as buildPythonPackage and buildPythonApplication no longer provide
<pkg>.overrideDerivation

Clean up the leftover of commit 58bfe74 ("buildPython*:
Deprecate and remove (buildPython* { ... }).override")
@ShamrockLee
Copy link
Contributor Author

@trofi I'll fix it as well.

Would you mind sharing how you found those evaluation errors in <pkg>.tests? The nixpkgs-review PR (Mic92/nixpkgs-review#397) to examine them hasn't been merged yet.

@ShamrockLee ShamrockLee force-pushed the build-python-local-override-overrideDerivation branch from a57edf0 to 296d1fd Compare October 22, 2024 06:26
buildPythonPackage no longer provide local .override attribute. Use
overridePythonAttrs instead.
@ShamrockLee ShamrockLee force-pushed the build-python-local-override-overrideDerivation branch from 296d1fd to 38e7722 Compare October 22, 2024 06:28
@trofi
Copy link
Contributor

trofi commented Oct 22, 2024

@trofi I'll fix it as well.

Would you mind sharing how you found those evaluation errors in <pkg>.tests? The nixpkgs-review PR (Mic92/nixpkgs-review#397) to examine them hasn't been merged yet.

I'm using this script: https://github.com/trofi/nixpkgs-overlays/blob/main/lib/all-attrs-iter.bash and run it as:

$ ./all-attrs-iter.bash -I nixpkgs=~/n --arg maxDepth 4 --arg verbose 3 --arg ignoreDrvAttrs false

~/n contains nixpkgs from staging branch. The caveat is that I have a bunch of patches against nixpkgs to skip other known eval failures.

@trofi
Copy link
Contributor

trofi commented Oct 23, 2024

Tested the attribute evaluation against the current branch state. All good now. Thank you!

@ShamrockLee
Copy link
Contributor Author

@trofi, could you officially approve this PR using the GitHub Review? This would make the approval more visible.

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-already-reviewed/2617/2063

@wegank wegank added the 12.approvals: 1 This PR was reviewed and approved by one reputable person label Oct 27, 2024
@ShamrockLee
Copy link
Contributor Author

As the Nixpkgs 24.11 branch-off is made, this PR should be backported onto the release-24.11 branch once it is merged.

@nix-owners nix-owners bot requested a review from mweinelt November 18, 2024 20:06
@ShamrockLee
Copy link
Contributor Author

Cc: @natsukagami

@natsukagami
Copy link
Contributor

@natsukium 👋

@ShamrockLee
Copy link
Contributor Author

@natsukium 👋

Oops! Sorry for bothering and thank you for the redirection.

Copy link
Member

@natsukium natsukium left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the missing. LGTM, thanks.

@natsukium natsukium merged commit 55f94d9 into NixOS:master Nov 19, 2024
31 checks passed
@natsukium natsukium added the backport release-24.11 Backport PR automatically label Nov 19, 2024
Copy link
Contributor

Copy link
Contributor

Git push to origin failed for release-24.11 with exitcode 1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 6.topic: python 8.has: changelog 8.has: documentation This PR adds or changes documentation 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 1-10 12.approvals: 1 This PR was reviewed and approved by one reputable person backport release-24.11 Backport PR automatically
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants