Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

opencv4 and scipy cause glibc version conflicts with cudaSupport #258327

Closed
RuRo opened this issue Oct 1, 2023 · 3 comments · Fixed by #265070
Closed

opencv4 and scipy cause glibc version conflicts with cudaSupport #258327

RuRo opened this issue Oct 1, 2023 · 3 comments · Fixed by #265070
Labels
0.kind: bug Something is broken

Comments

@RuRo
Copy link
Contributor

RuRo commented Oct 1, 2023

Describe the bug

Trying to import opencv4 and scipy with cuda can cause glibc version conflicts.

Steps To Reproduce

{
  inputs.nixpkgs.url = "github:nixos/nixpkgs/0396d3b0fb7f62ddc79a506ad3e6124f01d2ed0a";

  outputs = { nixpkgs, ... }: let
    reproduction = { pkgs, ... }: {
      nixpkgs.config = {
        allowUnfree = true;
        cudaSupport = true;

        # the cudaCapabilities line is probably not required for reproduction,
        # but I specify just a single capability version to reduce build time
        cudaCapabilities = [ "8.6" ];
      };

      environment.systemPackages = [(pkgs.python3.withPackages (ps: [
        ps.opencv4
        ps.scipy
      ]))];
    };
  in
  {
    nixosConfigurations.testing = nixpkgs.lib.nixosSystem {
      system = "x86_64-linux";
      modules = [
        { users.users.root.password = ""; }
        reproduction
      ];
    };
  };
}
nix build .#nixosConfigurations.testing.config.system.build.vm
result/bin/run-nixos-vm -nographic

Login as root (the password is empty) and then run

python -c "import cv2" # this is OK
python -c "import scipy.fft" # this is OK
python -c "import scipy.fft; import cv2" # this is also OK

python -c "import cv2; import scipy.fft" # Boom!
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/nix/store/5phig9ydshwkaka96xn6llxpfcsjyzrl-python3-3.10.12-env/lib/python3.10/site-packages/scipy/fft/__init__.py", line 92, in <module>
    from ._helper import next_fast_len
  File "/nix/store/5phig9ydshwkaka96xn6llxpfcsjyzrl-python3-3.10.12-env/lib/python3.10/site-packages/scipy/fft/_helper.py", line 3, in <module>
    from ._pocketfft import helper as _helper
  File "/nix/store/5phig9ydshwkaka96xn6llxpfcsjyzrl-python3-3.10.12-env/lib/python3.10/site-packages/scipy/fft/_pocketfft/__init__.py", line 3, in <module>
    from .basic import *
  File "/nix/store/5phig9ydshwkaka96xn6llxpfcsjyzrl-python3-3.10.12-env/lib/python3.10/site-packages/scipy/fft/_pocketfft/basic.py", line 6, in <module>
    from . import pypocketfft as pfft
ImportError: /nix/store/pfx4gg15nllsa6cwfhjnink1jr35dpfq-gcc-11.3.0-lib/lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /nix/store/5phig9ydshwkaka96xn6llxpfcsjyzrl-python3-3.10.12-env/lib/python3.10/site-package)

Expected behavior

No crashes

Notify maintainers

I am not really sure, which of the 3 components in the reproduction is actually responsible (if any), so I am pinging the maintainers of all three.

cuda: @ConnorBaker @samuela @SomeoneSerge @NixOS/cuda-maintainers
opencv4: @basvandijk
scipy: @FRidh @doronbehar

Also, this might be related to #220341 #226165

@RuRo RuRo added the 0.kind: bug Something is broken label Oct 1, 2023
@ConnorBaker
Copy link
Contributor

Yes, that looks to be another need for #226165 :(

Tangentially related — does GPU pass through work with NixOS VMs?

@RuRo
Copy link
Contributor Author

RuRo commented Oct 1, 2023

Tangentially related — does GPU pass through work with NixOS VMs?

I have no idea, sorry. I am using the VMs purely for creating minimal reproducible examples, not actually running stuff on them.

@SomeoneSerge
Copy link
Contributor

As per #226165 (comment), maybe you start seeing if it might suffice to use cudaPackages.stdenv instead of

python -c "import cv2; import scipy.fft" # Boom!

...awesome, I think we should add this to passthru.tests (note this doesn't require gpu access)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0.kind: bug Something is broken
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

3 participants