add release notes to 2.5.0 (#3360)

intel · Nov 5, 2024 · ecb3791 · ecb3791
1 parent a7f9cef
commit ecb3791
Show file tree

Hide file tree

Showing 47 changed files with 417 additions and 314 deletions.
diff --git a/cpu/2.5.0+cpu/_sources/tutorials/examples.md.txt b/cpu/2.5.0+cpu/_sources/tutorials/examples.md.txt
@@ -502,7 +502,7 @@ print("Execution finished")
 import torch
 from transformers import BertModel
 
-model = BertModel.from_pretrained("bert-base-uncased")
+model = BertModel.from_pretrained("bert-base-uncased", attn_implementation="eager")
 model.eval()
 
 vocab_size = model.config.vocab_size

diff --git a/cpu/2.5.0+cpu/_sources/tutorials/features/fast_bert.md.txt b/cpu/2.5.0+cpu/_sources/tutorials/features/fast_bert.md.txt
@@ -9,7 +9,7 @@ Currently `ipex.fast_bert` API is only well optimized for training. For inferenc
 
 ### Prerequisite
 
-- Transformers 4.6.0 ~ 4.43.2
+- Transformers 4.6.0 ~ 4.45.0
 
 ### Usage Example
 
@@ -20,7 +20,7 @@ An API `ipex.fast_bert` is provided for a simple usage. Usage of this API follow
 import torch
 from transformers import BertModel
 
-model = BertModel.from_pretrained("bert-base-uncased")
+model = BertModel.from_pretrained("bert-base-uncased", attn_implementation="eager")
 model.eval()
 
 vocab_size = model.config.vocab_size

diff --git a/cpu/2.5.0+cpu/_sources/tutorials/getting_started.md.txt b/cpu/2.5.0+cpu/_sources/tutorials/getting_started.md.txt
@@ -1,6 +1,6 @@
 # Quick Start
 
-The following instructions assume you have installed the Intel® Extension for PyTorch\*. For installation instructions, refer to [Installation](../../../index.html#installation?platform=cpu&version=main).
+The following instructions assume you have installed the Intel® Extension for PyTorch\*. For installation instructions, refer to [Installation](../../../index.html#installation?platform=cpu&version=v2.5.0%2Bcpu).
 
 To start using the Intel® Extension for PyTorch\* in your code, you need to make the following changes:
 
@@ -64,7 +64,6 @@ In [Cheat Sheet](./cheat_sheet.md), you can find more commands that can help you
 
 `ipex.llm.optimize` is used for Large Language Models (LLM).
 
-
 ```python
 import torch
 #################### code changes ####################  
@@ -157,4 +156,4 @@ with torch.inference_mode(), torch.cpu.amp.autocast(enabled=amp_enabled):
     print(gen_text, total_new_tokens, flush=True)
 ```
 
-More LLM examples, including usage of low precision data types are available in the [LLM Examples](https://github.com/intel/intel-extension-for-pytorch/tree/main/examples/cpu/llm) section.
+More LLM examples, including usage of low precision data types are available in the [LLM Examples](https://github.com/intel/intel-extension-for-pytorch/tree/release/2.5/examples/cpu/llm) section.
diff --git a/cpu/2.5.0+cpu/_sources/tutorials/installation.md.txt b/cpu/2.5.0+cpu/_sources/tutorials/installation.md.txt
@@ -1,8 +1,8 @@
 Installation
 ============
 
-Select your preferences and follow the installation instructions provided on the [Installation page](../../../index.html#installation?platform=cpu&version=v2.4.0%2Bcpu).
+Select your preferences and follow the installation instructions provided on the [Installation page](../../../index.html#installation?platform=cpu&version=v2.5.0%2Bcpu).
 
 After successful installation, refer to the [Quick Start](getting_started.md) and [Examples](examples.md) sections to start using the extension in your code.
 
-**NOTE:** For detailed instructions on installing and setting up the environment for Large Language Models (LLM), as well as example scripts, refer to the [LLM best practices](https://github.com/intel/intel-extension-for-pytorch/tree/v2.4.0%2Bcpu/examples/cpu/llm).
+**NOTE:** For detailed instructions on installing and setting up the environment for Large Language Models (LLM), as well as example scripts, refer to the [LLM best practices](https://github.com/intel/intel-extension-for-pytorch/tree/v2.5.0%2Bcpu/examples/cpu/llm).
diff --git a/cpu/2.5.0+cpu/_sources/tutorials/introduction.rst.txt b/cpu/2.5.0+cpu/_sources/tutorials/introduction.rst.txt
@@ -16,7 +16,7 @@ the `Large Language Models (LLM) <llm.html>`_ section.
 
 Get Started
 -----------
-- `Installation <../../../index.html#installation?platform=cpu&version=v2.4.0%2Bcpu>`_
+- `Installation <../../../index.html#installation?platform=cpu&version=v2.5.0%2Bcpu>`_
 - `Quick Start <getting_started.md>`_
 - `Examples <examples.md>`_
 

diff --git a/cpu/2.5.0+cpu/_sources/tutorials/releases.md.txt b/cpu/2.5.0+cpu/_sources/tutorials/releases.md.txt
@@ -1,6 +1,33 @@
 Releases
 ========
 
+## 2.5.0
+
+We are excited to announce the release of Intel® Extension for PyTorch* 2.5.0+cpu which accompanies PyTorch 2.5. This release mainly brings you the support for Llama3.2, optimization on newly launched Intel® Xeon® 6 P-core platform, GPTQ/AWQ format support, and latest optimization to push better performance for LLM models. This release also includes a set of bug fixing and small optimizations. We want to sincerely thank our dedicated community for your contributions. As always, we encourage you to try this release and feedback as to improve further on this product.
+
+### Highlights
+
+* Llama 3.2 support
+
+Meta has newly released [Llama 3.2](https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices/), which includes small and medium-sized vision LLMs (11B and 90B), and lightweight, text-only models (1B and 3B). Intel® Extension for PyTorch* provides [support of Llama 3.2](https://www.intel.com/content/www/us/en/developer/articles/technical/intel-ai-solutions-support-the-new-llama-3-2-model.html) since its launch date with early release version, and now support with this official release.
+
+* Optimization for Intel® Xeon® 6
+Intel® Xeon® 6 deliver new degrees of performance with more cores, a choice of microarchitecture, additional memory bandwidth, and exceptional input/output (I/O) across a range of workloads. Intel® Extension for PyTorch* provides dedicated optimization on this new processor family for features like Multiplexed Rank DIMM (MRDIMM), SNC=3 scenario, etc..
+
+* Large Language Model (LLM) optimization:
+Intel® Extension for PyTorch* provides more feature support of the weight only quantization including GPTQ/AWQ format support, symmetric quantization of activation and weight, and added chunked prefill/prefix prefill support in LLM module API, etc.. These features enable better adoption of community model weight and provides better performance for low-precision scenarios. This release also extended the optimized models to include newly published Llama 3.2 vision models. A full list of optimized models can be found at [LLM optimization](https://github.com/intel/intel-extension-for-pytorch/tree/v2.5.0+cpu/examples/cpu/llm/inference).
+
+* Bug fixing and other optimization
+    - Optimized the performance of the IndirectAccessKVCacheAttention kernel
+[#3185](https://github.com/intel/intel-extension-for-pytorch/commit/8572e1faf97998783ea2a7fc6ee3094090feebc4) [#3209](https://github.com/intel/intel-extension-for-pytorch/commit/65e96630a2e17f7b762c5c765f10264ad08db098) [#3214](https://github.com/intel/intel-extension-for-pytorch/commit/a04214f7ab4e43648d75abdcf0fae53e5076be2b) [#3218](https://github.com/intel/intel-extension-for-pytorch/commit/f219012ab1babbc67c9b545fa7251cd981a2a3a2) [#3248](https://github.com/intel/intel-extension-for-pytorch/commit/9f6178eb028d36b3ed1f5985e57b7cf160acf38a)
+    - Fixed the Segmentation fault in the IndirectAccessKVCacheAttention kernel [#3246](https://github.com/intel/intel-extension-for-pytorch/commit/bee5ab644086c9b25eb61916c6773932c74667d3)
+    - Fixed the correctness issue in the PagedAttention kernel for Llama-68M-Chat-v1 [#3307](https://github.com/intel/intel-extension-for-pytorch/commit/638a7d26acb33af450ea9869b5b43ccdbe0e962b)
+    - Fixed the support in `ipex.llm.optimize` to ensure `model.generate` returns the correct output type when `return_dict_in_generate` is set to `True`. [#3333](https://github.com/intel/intel-extension-for-pytorch/commit/584a4e2e2c6193b926554f951d2608489cac5d7a)
+    - Optimized the performance of the Flash Attention kernel [#3291](https://github.com/intel/intel-extension-for-pytorch/commit/8fb43ec45ed93b62efef07f4b2e8dcd7dd502b8b)
+    - Upgraded oneDNN to v3.6 [#3305](https://github.com/intel/intel-extension-for-pytorch/commit/91639fa0812ee3c12c672002c2bf5cf1cac4bc0a)
+
+**Full Changelog**: https://github.com/intel/intel-extension-for-pytorch/compare/v2.4.0+cpu...v2.5.0+cpu
+
 ## 2.4.0
 
 We are excited to announce the release of Intel® Extension for PyTorch\* 2.4.0+cpu which accompanies PyTorch 2.4. This release mainly brings you the support for Llama3.1, basic support for LLM serving frameworks like vLLM/TGI, and a set of optimization to push better performance for LLM models. This release also extends the list of optimized LLM models to a broader level and includes a set of bug fixing and small optimizations. We want to sincerely thank our dedicated community for your contributions. As always, we encourage you to try this release and feedback as to improve further on this product.

diff --git a/cpu/2.5.0+cpu/_static/htmls/tbl_deepspeed.html b/cpu/2.5.0+cpu/_static/htmls/tbl_deepspeed.html
@@ -44,6 +44,18 @@
       <td><p style="text-align: center; vertical-align: middle;">🟩</p></td>
       <td><p style="text-align: center; vertical-align: middle;">🟩</p></td>
     </tr>
+    <tr class="row-even">
+      <td><p>LLAMA</p></td>
+      <td><p>meta-llama/Llama-3.2-3B-Instruct</p></td>
+      <td><p style="text-align: center; vertical-align: middle;">🟩</p></td>
+      <td><p style="text-align: center; vertical-align: middle;">🟩</p></td>
+    </tr>
+    <tr class="row-odd">
+      <td><p>LLAMA</p></td>
+      <td><p>meta-llama/Llama-3.2-11B-Vision-Instruct</p></td>
+      <td><p style="text-align: center; vertical-align: middle;">🟩</p></td>
+      <td><p style="text-align: center; vertical-align: middle;">🟩</p></td>
+    </tr>
     <tr class="row-even">
       <td><p>GPT-J</p></td>
       <td><p>EleutherAI/gpt-j-6b</p></td>
@@ -53,13 +65,13 @@
     <tr class="row-odd">
       <td><p>GPT-NEOX</p></td>
       <td><p>EleutherAI/gpt-neox-20b</p></td>
-      <td><p style="text-align: center; vertical-align: middle;">🟨</p></td>
+      <td><p style="text-align: center; vertical-align: middle;">🟩</p></td>
       <td><p style="text-align: center; vertical-align: middle;">🟩</p></td>
     </tr>
     <tr class="row-even">
       <td><p>DOLLY</p></td>
       <td><p>databricks/dolly-v2-12b</p></td>
-      <td><p style="text-align: center; vertical-align: middle;">🟨</p></td>
+      <td><p style="text-align: center; vertical-align: middle;">🟩</p></td>
       <td><p style="text-align: center; vertical-align: middle;">🟩</p></td>
     </tr>
     <tr class="row-odd">
@@ -77,7 +89,7 @@
     <tr class="row-odd">
       <td><p>OPT</p></td>
       <td><p>facebook/opt-30b</p></td>
-      <td><p style="text-align: center; vertical-align: middle;">🟨</p></td>
+      <td><p style="text-align: center; vertical-align: middle;">🟩</p></td>
       <td><p style="text-align: center; vertical-align: middle;">🟩</p></td>
     </tr>
     <tr class="row-even">
@@ -89,7 +101,7 @@
     <tr class="row-odd">
       <td><p>Bloom</p></td>
       <td><p>bigscience/bloom-1b7</p></td>
-      <td><p style="text-align: center; vertical-align: middle;">🟨</p></td>
+      <td><p style="text-align: center; vertical-align: middle;">🟩</p></td>
       <td><p style="text-align: center; vertical-align: middle;">🟩</p></td>
     </tr>
     <tr class="row-even">
@@ -113,7 +125,7 @@
     <tr class="row-odd">
       <td><p>Baichuan</p></td>
       <td><p>baichuan-inc/Baichuan-13B-Chat</p></td>
-      <td><p style="text-align: center; vertical-align: middle;">🟨</p></td>
+      <td><p style="text-align: center; vertical-align: middle;">🟩</p></td>
       <td><p style="text-align: center; vertical-align: middle;">🟩</p></td>
     </tr>
     <tr class="row-even">
@@ -207,8 +219,4 @@
       <td><p style="text-align: center; vertical-align: middle;">🟩</p></td>
     </tr>
   </tbody>
-</table>
-<ul class="simple">
-  <li><p>🟩 signifies that the model can perform well and with good accuracy (&lt;1% difference as compared with FP32).</p></li>
-  <li><p>🟨 signifies that the model can perform well while accuracy may not been in a perfect state (&gt;1% difference as compared with FP32).</p></li>
-</ul>
+</table>