diff --git a/docs/source/_toctree.yml b/docs/source/_toctree.yml index fdfe19ee4..a72eb1967 100644 --- a/docs/source/_toctree.yml +++ b/docs/source/_toctree.yml @@ -12,6 +12,8 @@ title: 8-bit optimizers - local: algorithms title: Algorithms + - local: non_cuda_backends + title: Non-CUDA compute backends - local: fsdp_qlora title: FSDP-QLoRA - local: integrations diff --git a/docs/source/non_cuda_backends.mdx b/docs/source/non_cuda_backends.mdx new file mode 100644 index 000000000..fca586534 --- /dev/null +++ b/docs/source/non_cuda_backends.mdx @@ -0,0 +1,27 @@ +# Multi-backend support (non-CUDA backends) + +As part of a recent refactoring effort, we will soon offer official multi-backend support. Currently, this feature is available in a preview alpha release, allowing us to gather early feedback from users to improve the functionality and identify any bugs. + +At present, the Intel CPU and AMD ROCm backends are considered fully functional. The Intel XPU backend has limited functionality and is less mature. + +Please refer to the [installation instructions](./installation#multi-backend) for details on installing the backend you intend to test (and hopefully provide feedback on). + +> [!Tip] +> Apple Silicon support is planned for Q4 2024. We are actively seeking contributors to help implement this, develop a concrete plan, and create a detailed list of requirements. Due to limited resources, we rely on community contributions for this implementation effort. To discuss further, please spell out your thoughts and discuss in [this GitHub discussion](https://github.com/bitsandbytes-foundation/bitsandbytes/discussions/1340) and tag `@Titus-von-Koeller` and `@matthewdouglas`. Thank you! + +## Alpha Release + +As we are currently in the alpha testing phase, bugs are expected, and performance might not meet expectations. However, this is exactly what we want to discover from **your** perspective as the end user! + +Please share and discuss your feedback with us here: + +- [Github Discussion: Multi-backend refactor: Alpha release ( AMD ROCm ONLY )](https://github.com/bitsandbytes-foundation/bitsandbytes/discussions/1339) +- [Github Discussion: Multi-backend refactor: Alpha release ( Intel ONLY )](https://github.com/bitsandbytes-foundation/bitsandbytes/discussions/1338) + +Thank you for your support! + +## Benchmarks + +### Intel + +### AMD