CUDA Setup failed despite GPU being available.

[SrKium]

System Info

windowsServer2022
E5-2686v4
RTX2080 Ti

Reproduction

===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues

binary_path: C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\bitsandbytes\cuda_setup\libbitsandbytes_cuda116.dll
CUDA SETUP: Loading binary C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\bitsandbytes\cuda_setup\libbitsandbytes_cuda116.dll...
Could not find module 'C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\bitsandbytes\cuda_setup\libbitsandbytes_cuda116.dll' (or one of its dependencies). Try using the full path with constructor syntax.
CUDA SETUP: Loading binary C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\bitsandbytes\cuda_setup\libbitsandbytes_cuda116.dll...
Could not find module 'C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\bitsandbytes\cuda_setup\libbitsandbytes_cuda116.dll' (or one of its dependencies). Try using the full path with constructor syntax.
CUDA SETUP: Loading binary C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\bitsandbytes\cuda_setup\libbitsandbytes_cuda116.dll...
Could not find module 'C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\bitsandbytes\cuda_setup\libbitsandbytes_cuda116.dll' (or one of its dependencies). Try using the full path with constructor syntax.
CUDA SETUP: Problem: The main issue seems to be that the main CUDA library was not detected.
CUDA SETUP: Solution 1): Your paths are probably not up-to-date. You can update them via: sudo ldconfig.
CUDA SETUP: Solution 2): If you do not have sudo rights, you can do the following:
CUDA SETUP: Solution 2a): Find the cuda library via: find / -name libcuda.so 2>/dev/null
CUDA SETUP: Solution 2b): Once the library is found add it to the LD_LIBRARY_PATH: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:FOUND_PATH_FROM_2a
CUDA SETUP: Solution 2c): For a permanent solution add the export from 2b into your .bashrc file, located at ~/.bashrc
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 187, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 146, in _get_module_details
return _get_module_details(pkg_main_name, error)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 110, in get_module_details
import(pkg_name)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\bitsandbytes_init.py", line 7, in
from .autograd.functions import (
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\bitsandbytes\autograd_init.py", line 1, in
from ._functions import undo_layout, get_inverse_transform_indices
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\bitsandbytes\autograd_functions.py", line 9, in
import bitsandbytes.functional as F
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\bitsandbytes\functional.py", line 17, in
from .cextension import COMPILED_WITH_CUDA, lib
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\bitsandbytes\cextension.py", line 22, in
raise RuntimeError('''
RuntimeError:
CUDA Setup failed despite GPU being available. Inspect the CUDA SETUP outputs above to fix your environment!
If you cannot find any issues and suspect a bug, please open an issue with detals about your environment:
https://github.com/TimDettmers/bitsandbytes/issues

Expected behavior

I just want it to work!

[matthewdouglas]

Hi @SrKium,
Can you please let us know what version of PyTorch and bitsandbytes you are using? Based on the output, it is looking like you may be using PyTorch 1.13 or earlier. We no longer ship CUDA 11.6 binaries, and generally, I would recommend updating to a minimum PyTorch >= 2.0.

Alternatively, you may be able to work around this for a short while by setting an env var BNB_CUDA_VERSION=118, but we're unlikely to continue to support torch < 2.