torch-scatter gives RuntimeError: Not compiled with CUDA support

[maes21353]

I am getting the following error from the torch-scatter:

RuntimeError: Not compiled with CUDA support

I have tried multiple solutions suggested on this channel, but none worked.

I am currently running on torch= 2.0.1+cu117.

Things that I have tried when installing the package:

• pip install --no-cache-dir torch-scatter==2.1.1 -f https://data.pyg.org/whl/torch-2.0.1+cu117.html
• pip install torch-scatter==2.1.1 -f https://data.pyg.org/whl/torch-2.0.1+cu117.html
• pip install torch-scatter https://data.pyg.org/whl/torch-2.0.1+cu117.html
• pip install torch-scatter

Before each of the above installations, I was first uninstalling torch-scatter.
Does anyone know why this is happening?

[rusty1s]

What happens if you install via

FORCE_CUDA=1 pip install torch-scatter

[maes21353]

What happens if you install via

FORCE_CUDA=1 pip install torch-scatter

Used

FORCE_CUDA=1 pip install --no-cache-dir torch-scatter

instead, and got the following error:

error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [14 lines of output]
Traceback (most recent call last):
File "", line 2, in
File "", line 34, in
File "/tmp/pip-install-s706li4b/torch-scatter_8f31c49792df42c9bb54c710eb16077e/setup.py", line 134, in
ext_modules=get_extensions() if not BUILD_DOCS else [],
File "/tmp/pip-install-s706li4b/torch-scatter_8f31c49792df42c9bb54c710eb16077e/setup.py", line 94, in get_extensions
extension = Extension(
File "/venv/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1048, in CUDAExtension
library_dirs += library_paths(cuda=True)
File "/venv/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1179, in library_paths
if (not os.path.exists(_join_cuda_home(lib_dir)) and
File "/venv/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 2223, in _join_cuda_home
raise EnvironmentError('CUDA_HOME environment variable is not set. '
OSError: CUDA_HOME environment variable is not set. Please set it to your CUDA install root.
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.

[rusty1s]

Can you set the CUDA_HOME path to your CUDA installation and try again?

[maes21353]

Can you set the CUDA_HOME path to your CUDA installation and try again?

Still getting an error. This is the full message (sorry for the lengthy copy)

× python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [63 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-3.10
      creating build/lib.linux-x86_64-3.10/torch_scatter
      copying torch_scatter/__init__.py -> build/lib.linux-x86_64-3.10/torch_scatter
      copying torch_scatter/segment_coo.py -> build/lib.linux-x86_64-3.10/torch_scatter
      copying torch_scatter/testing.py -> build/lib.linux-x86_64-3.10/torch_scatter
      copying torch_scatter/scatter.py -> build/lib.linux-x86_64-3.10/torch_scatter
      copying torch_scatter/utils.py -> build/lib.linux-x86_64-3.10/torch_scatter
      copying torch_scatter/segment_csr.py -> build/lib.linux-x86_64-3.10/torch_scatter
      copying torch_scatter/placeholder.py -> build/lib.linux-x86_64-3.10/torch_scatter
      creating build/lib.linux-x86_64-3.10/torch_scatter/composite
      copying torch_scatter/composite/__init__.py -> build/lib.linux-x86_64-3.10/torch_scatter/composite
      copying torch_scatter/composite/std.py -> build/lib.linux-x86_64-3.10/torch_scatter/composite
      copying torch_scatter/composite/softmax.py -> build/lib.linux-x86_64-3.10/torch_scatter/composite
      copying torch_scatter/composite/logsumexp.py -> build/lib.linux-x86_64-3.10/torch_scatter/composite
      running egg_info
      writing torch_scatter.egg-info/PKG-INFO
      writing dependency_links to torch_scatter.egg-info/dependency_links.txt
      writing requirements to torch_scatter.egg-info/requires.txt
      writing top-level names to torch_scatter.egg-info/top_level.txt
      reading manifest file 'torch_scatter.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      warning: no previously-included files matching '*' found under directory 'test'
      adding license file 'LICENSE'
      writing manifest file 'torch_scatter.egg-info/SOURCES.txt'
      running build_ext
      /venv/lib/python3.10/site-packages/torch/utils/cpp_extension.py:388: UserWarning: The detected CUDA version (11.5) has a minor version mismatch with the version that was used to compile PyTorch (11.7). Most likely this shouldn't be a problem.
        warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda))
      building 'torch_scatter._segment_csr_cuda' extension
      creating build/temp.linux-x86_64-3.10
      creating build/temp.linux-x86_64-3.10/csrc
      creating build/temp.linux-x86_64-3.10/csrc/cpu
      creating build/temp.linux-x86_64-3.10/csrc/cuda
      x86_64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DWITH_PYTHON -DWITH_CUDA -Icsrc -I/venv/lib/python3.10/site-packages/torch/include -I/venv/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/venv/lib/python3.10/site-packages/torch/include/TH -I/venv/lib/python3.10/site-packages/torch/include/THC -I/usr/lib/nvidia-cuda-toolkit/include -I/venv/include -I/usr/include/python3.10 -c csrc/cpu/segment_csr_cpu.cpp -o build/temp.linux-x86_64-3.10/csrc/cpu/segment_csr_cpu.o -O3 -Wno-sign-compare -DAT_PARALLEL_OPENMP -fopenmp -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=_segment_csr_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17
      /venv/lib/python3.10/site-packages/torch/cuda/__init__.py:546: UserWarning: Can't initialize NVML
        warnings.warn("Can't initialize NVML")
      /usr/lib/nvidia-cuda-toolkit/bin/nvcc -DWITH_PYTHON -DWITH_CUDA -Icsrc -I/venv/lib/python3.10/site-packages/torch/include -I/venv/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/venv/lib/python3.10/site-packages/torch/include/TH -I/venv/lib/python3.10/site-packages/torch/include/THC -I/usr/lib/nvidia-cuda-toolkit/include -I/venv/include -I/usr/include/python3.10 -c csrc/cuda/segment_csr_cuda.cu -o build/temp.linux-x86_64-3.10/csrc/cuda/segment_csr_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -O3 --expt-relaxed-constexpr -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=_segment_csr_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 -std=c++17
      /venv/lib/python3.10/site-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
                detected during:
                  instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
      (61): here
                  instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]
      /venv/lib/python3.10/site-packages/torch/include/c10/core/TensorImpl.h(77): here
      
venv/lib/python3.10/site-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
                detected during:
                  instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
      (61): here
                  instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
      /venv/lib/python3.10/site-packages/torch/include/ATen/core/qualified_name.h(73): here
      
      /usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
        435 |         function(_Functor&& __f)
            |                                                                                                                                                 ^
      /usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
      /usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
        530 |         operator=(_Functor&& __f)
            |                                                                                                                                                  ^
      /usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
      error: command '/usr/lib/nvidia-cuda-toolkit/bin/nvcc' failed with exit code 1
      [end of output]
    note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for torch-scatter

[rusty1s]

Mh, I am not totally sure TBH. Never seen this error before. What is the gcc version you are compiling with (gcc --version)? Also, can you post the full log of installing (FORCE_CUDA=1 pip install --verbose torch-scatter)? There is also this warning

The detected CUDA version (11.5) has a minor version mismatch with the version that was used to compile PyTorch (11.7). Most likely this shouldn't be a problem.

which may be related to this issue.

[travisddavies]

I'm getting this exact same issue

[ogutsevda]

Me too!

[YuanYunshuang]

Using CUDA11.8, scatter_add works but scatter_max shows this error