Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Build] Add CUDA 12.6.2 build; update 12.5.0 to 12.5.1 #1431

Merged
merged 3 commits into from
Dec 2, 2024
Merged

[Build] Add CUDA 12.6.2 build; update 12.5.0 to 12.5.1 #1431

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7bce4cb
[Build] Add CUDA 12.6.2 build; update 12.5.0 to 12.5.1
matthewdouglas Nov 29, 2024
d6c9b9d
bump cuda-toolkit action version
matthewdouglas Nov 29, 2024
88f9a80
Update docs for cuda versions
matthewdouglas Nov 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/python-package.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ jobs:
os: [ubuntu-latest, windows-latest]
arch: [x86_64, aarch64]
cuda_version:
["11.7.1", "11.8.0", "12.0.1", "12.1.1", "12.2.2", "12.3.2", "12.4.1", "12.5.0"]
["11.7.1", "11.8.0", "12.0.1", "12.1.1", "12.2.2", "12.3.2", "12.4.1", "12.5.1", "12.6.2"]
exclude:
- os: windows-latest # This probably requires arm64 Windows agents
arch: aarch64
Expand All @@ -78,7 +78,7 @@ jobs:
if: startsWith(matrix.os, 'ubuntu')
uses: docker/setup-qemu-action@v2
# Windows: We install Cuda on the agent (slow)
- uses: Jimver/[email protected].16
- uses: Jimver/[email protected].19
if: startsWith(matrix.os, 'windows')
id: cuda-toolkit
with:
Expand Down
16 changes: 8 additions & 8 deletions docs/source/installation.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Welcome to the installation guide for the `bitsandbytes` library! This document

## CUDA[[cuda]]

`bitsandbytes` is currently only supported on CUDA GPUs for CUDA versions **11.0 - 12.5**. However, there's an ongoing multi-backend effort under development, which is currently in alpha. If you're interested in providing feedback or testing, check out [the multi-backend section below](#multi-backend).
`bitsandbytes` is currently only supported on CUDA GPUs for CUDA versions **11.0 - 12.6**. However, there's an ongoing multi-backend effort under development, which is currently in alpha. If you're interested in providing feedback or testing, check out [the multi-backend section below](#multi-backend).

### Supported CUDA Configurations[[cuda-pip]]

Expand All @@ -29,7 +29,7 @@ The latest version of `bitsandbytes` builds on the following configurations:
|-------------|------------------|----------------------|
| **Linux** | 11.7 - 12.3 | GCC 11.4 |
| | 12.4+ | GCC 13.2 |
| **Windows** | 11.7 - 12.4 | MSVC 19.38+ (VS2022) |
| **Windows** | 11.7 - 12.6 | MSVC 19.38+ (VS2022) |

For Linux systems, ensure your hardware meets the following requirements:

Expand Down Expand Up @@ -115,7 +115,7 @@ pip install -e . # `-e` for "editable" install, when developing BNB (otherwise

Windows systems require Visual Studio with C++ support as well as an installation of the CUDA SDK.

To compile from source, you need CMake >= **3.22.1** and Python >= **3.8** installed. You should also install CUDA Toolkit by following the [CUDA Installation Guide for Windows](https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/index.html) guide from NVIDIA.
To compile from source, you need CMake >= **3.22.1** and Python >= **3.9** installed. You should also install CUDA Toolkit by following the [CUDA Installation Guide for Windows](https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/index.html) guide from NVIDIA.

Refer to the following table if you're using another CUDA Toolkit version.

Expand Down Expand Up @@ -150,12 +150,12 @@ Then locally install the CUDA version you need with this script from bitsandbyte
```bash
wget https://raw.githubusercontent.com/bitsandbytes-foundation/bitsandbytes/main/install_cuda.sh
# Syntax cuda_install CUDA_VERSION INSTALL_PREFIX EXPORT_TO_BASH
# CUDA_VERSION in {110, 111, 112, 113, 114, 115, 116, 117, 118, 120, 121, 122, 123, 124, 125}
# CUDA_VERSION in {110, 111, 112, 113, 114, 115, 116, 117, 118, 120, 121, 122, 123, 124, 125, 126}
# EXPORT_TO_BASH in {0, 1} with 0=False and 1=True

# For example, the following installs CUDA 11.7 to ~/local/cuda-11.7 and exports the path to your .bashrc
# For example, the following installs CUDA 12.6 to ~/local/cuda-12.6 and exports the path to your .bashrc

bash install_cuda.sh 117 ~/local 1
bash install_cuda.sh 126 ~/local 1
```

2. Set the environment variables `BNB_CUDA_VERSION` and `LD_LIBRARY_PATH` by manually overriding the CUDA version installed by PyTorch.
Expand All @@ -171,8 +171,8 @@ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<PATH>
For example, to use a local install path:

```bash
export BNB_CUDA_VERSION=117
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/YOUR_USERNAME/local/cuda-11.7
export BNB_CUDA_VERSION=126
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/YOUR_USERNAME/local/cuda-12.6
```

3. Now when you launch bitsandbytes with these environment variables, the PyTorch CUDA version is overridden by the new CUDA version (in this example, version 11.7) and a different bitsandbytes library is loaded.
Expand Down
3 changes: 2 additions & 1 deletion install_cuda.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@
"122": "https://developer.download.nvidia.com/compute/cuda/12.2.2/local_installers/cuda_12.2.2_535.104.05_linux.run",
"123": "https://developer.download.nvidia.com/compute/cuda/12.3.2/local_installers/cuda_12.3.2_545.23.08_linux.run",
"124": "https://developer.download.nvidia.com/compute/cuda/12.4.1/local_installers/cuda_12.4.1_550.54.15_linux.run",
"125": "https://developer.download.nvidia.com/compute/cuda/12.5.0/local_installers/cuda_12.5.0_555.42.02_linux.run",
"125": "https://developer.download.nvidia.com/compute/cuda/12.5.1/local_installers/cuda_12.5.1_555.42.06_linux.run",
"126": "https://developer.download.nvidia.com/compute/cuda/12.6.2/local_installers/cuda_12.6.2_560.35.03_linux.run",
}


Expand Down
6 changes: 5 additions & 1 deletion install_cuda.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@ URL121=https://developer.download.nvidia.com/compute/cuda/12.1.1/local_installer
URL122=https://developer.download.nvidia.com/compute/cuda/12.2.2/local_installers/cuda_12.2.2_535.104.05_linux.run
URL123=https://developer.download.nvidia.com/compute/cuda/12.3.2/local_installers/cuda_12.3.2_545.23.08_linux.run
URL124=https://developer.download.nvidia.com/compute/cuda/12.4.1/local_installers/cuda_12.4.1_550.54.15_linux.run
URL125=https://developer.download.nvidia.com/compute/cuda/12.5.0/local_installers/cuda_12.5.0_555.42.02_linux.run
URL125=https://developer.download.nvidia.com/compute/cuda/12.5.1/local_installers/cuda_12.5.1_555.42.06_linux.run
URL126=https://developer.download.nvidia.com/compute/cuda/12.6.2/local_installers/cuda_12.6.2_560.35.03_linux.run

CUDA_VERSION=$1
BASE_PATH=$2
Expand Down Expand Up @@ -64,6 +65,9 @@ if [[ -n "$CUDA_VERSION" ]]; then
elif [[ "$CUDA_VERSION" -eq "125" ]]; then
URL=$URL125
FOLDER=cuda-12.5
elif [[ "$CUDA_VERSION" -eq "126" ]]; then
URL=$URL126
FOLDER=cuda-12.6
else
echo "argument error: No cuda version passed as input. Choose among versions 110 to 125"
fi
Expand Down