From aca80a06b218283c98d17963d231b2a0578b7b6a Mon Sep 17 00:00:00 2001 From: Titus <9048635+Titus-von-Koeller@users.noreply.github.com> Date: Wed, 21 Feb 2024 18:51:45 +0100 Subject: [PATCH] bf16: change to warning --- docs/source/integrations.mdx | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/source/integrations.mdx b/docs/source/integrations.mdx index 565156513..bd9f4641b 100644 --- a/docs/source/integrations.mdx +++ b/docs/source/integrations.mdx @@ -6,8 +6,10 @@ Please review the [bitsandbytes section in the Accelerate docs](https://huggingf Details about the BitsAndBytesConfig can be found [here](https://huggingface.co/docs/transformers/v4.37.2/en/main_classes/quantization#transformers.BitsAndBytesConfig). -## Beware: bf16 is the optimal compute data type -If your hardware supports it, `bf16` is the optimal compute dtype. The default is `float32` for backward compatibility and numerical stability. `float16` often leads to numerical instabilities, but `bfloat16` provides the benefits of both worlds: numerical stability and significant computation speedup. Therefore, be sure to check if your hardware supports `bf16` and configure it using the `bnb_4bit_compute_dtype` parameter in BitsAndBytesConfig: +> [!WARNING] +> **Beware: bf16 is the optimal compute data type!** +> +> If your hardware supports it, `bf16` is the optimal compute dtype. The default is `float32` for backward compatibility and numerical stability. `float16` often leads to numerical instabilities, but `bfloat16` provides the benefits of both worlds: numerical stability and significant computation speedup. Therefore, be sure to check if your hardware supports `bf16` and configure it using the `bnb_4bit_compute_dtype` parameter in BitsAndBytesConfig: ```py import torch