update 2.1.10 performance numbers (#2652)

intel · Mar 6, 2024 · 38d7020 · 38d7020
1 parent d09f53e
commit 38d7020
Show file tree

Hide file tree

Showing 63 changed files with 510 additions and 59 deletions.
diff --git a/xpu/2.1.10+xpu/_images/four_card.png b/xpu/2.1.10+xpu/_images/four_card.png
diff --git a/xpu/2.1.10+xpu/_images/single_card.png b/xpu/2.1.10+xpu/_images/single_card.png
diff --git a/xpu/2.1.10+xpu/_images/single_tile.png b/xpu/2.1.10+xpu/_images/single_tile.png
diff --git a/xpu/2.1.10+xpu/_images/two_card.png b/xpu/2.1.10+xpu/_images/two_card.png
diff --git a/xpu/2.1.10+xpu/_sources/index.rst.txt b/xpu/2.1.10+xpu/_sources/index.rst.txt
@@ -59,6 +59,7 @@ The team tracks bugs and enhancement requests using `GitHub issues <https://gith
    tutorials/introduction
    tutorials/features
    Large Language Models (LLM)<tutorials/llm>
+   tutorials/performance
    tutorials/technical_details
    tutorials/releases
    tutorials/performance_tuning/known_issues

diff --git a/xpu/2.1.10+xpu/_sources/tutorials/performance.md.txt b/xpu/2.1.10+xpu/_sources/tutorials/performance.md.txt
@@ -0,0 +1,73 @@
+Performance
+===========
+
+## Overview
+
+This page shows performance boost with Intel® Extension for PyTorch\* on several popular topologies.
+
+## Performance Data for Intel® AI Data Center Products
+
+Find the latest performance data for Intel® Data Center Max 1550 GPU, including detailed hardware and software configurations.
+
+## LLM Performance
+
+We benchmarked GPT-J 6B, LLaMA2 7B, 13B, OPT 6.7B, Bloom-7B with test input token length set to 1024. The datatype is FP16 for all the models.
+
+![Single Tile](../images/performancev2.1.10/single_tile.png)
+
+![Single Card](../images/performancev2.1.10/single_card.png)
+
+![Two Card](../images/performancev2.1.10/two_card.png)
+
+![Four Card](../images/performancev2.1.10/four_card.png)
+
+
+
+### Configuration
+
+#### Software Version
+
+| Software | Version |
+| :-: | :-: |
+| PyTorch | [v2.1](https://pytorch.org/get-started/locally/) |
+| Intel® Extension for PyTorch\* | [v2.1.10+xpu](https://github.com/intel/intel-extension-for-pytorch/releases) |
+| Intel® oneAPI Base Toolkit | [2024.0](https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit-download.html) |
+| Torch-CCL | [2.1.100](https://github.com/intel/torch-ccl/tree/ccl_torch2.1.100%2Bxpu) |
+| GPU Driver |  [736.25](https://dgpu-docs.intel.com/releases/stable_736_25_20231031.html) |
+| Transformers | [v4.31.0](https://github.com/huggingface/transformers/tree/v4.31.0) |  
+| DeepSpeed | [commit 4fc181b0](https://github.com/microsoft/DeepSpeed/tree/4fc181b0)  |  
+| Intel® Extension for DeepSpeed\* | [commit ec33277](https://github.com/intel/intel-extension-for-deepspeed/tree/ec33277)  | 
+
+
+#### Hardware Configuration
+
+
+CPU Configuration: 
+
+| CPU | Intel(R) Xeon(R) Platinum 8480+ CPU  | 
+| :-: | :-: | 
+| Number of nodes | 1 | 
+| Number of sockets | 2 | 
+| Cores/Socket | 56 |
+| Threads/Core | 2 | 
+| uCode | 0x2b0004b1 | 
+| Hyper-Threading | ON | 
+| TurboBoost | ON | 
+| BIOS version | SE5C7411.86B.9525.D25.2304190630 |  
+| Number of DDR Memory slots | 16 | 
+| Capacity of DDR memory per slot | 64GB | 
+| DDR frequency | 4800 | 
+| Total Memory/Node (DDR+DCPMM) | 1024GB | 
+| Host OS | Ubuntu 22.04.3 LTS | 
+| Host Kernel | 5.17.0-1020-oem | 
+| [Spectre-Meltdown Mitigation](https://github.com/speed47/spectre-meltdown-checker) | Mitigated | 
+
+Single tile of 4X PVC OAM Configuration: 
+
+| GPU | Intel(R) Data Center Max 1550 GPU | 
+| :-: | :-: | 
+| IFWI | PVC.PS.B4.P.Si.2023.WW42.3\_25MHzi\_Quad\_DAMeni\_OAM600W\_IFRv2332i\_PSCnull\_IFWI.bin | 
+| ECC | ON |
+| AMC SW | AMC FW 6.2 | 
+| Precision | FP16 | 
+
diff --git a/xpu/2.1.10+xpu/genindex.html b/xpu/2.1.10+xpu/genindex.html
@@ -53,6 +53,7 @@
 <li class="toctree-l1"><a class="reference internal" href="tutorials/introduction.html">Introduction</a></li>
 <li class="toctree-l1"><a class="reference internal" href="tutorials/features.html">Features</a></li>
 <li class="toctree-l1"><a class="reference internal" href="tutorials/llm.html">Large Language Models (LLM)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tutorials/performance.html">Performance</a></li>
 <li class="toctree-l1"><a class="reference internal" href="tutorials/technical_details.html">Technical Details</a></li>
 <li class="toctree-l1"><a class="reference internal" href="tutorials/releases.html">Releases</a></li>
 <li class="toctree-l1"><a class="reference internal" href="tutorials/performance_tuning/known_issues.html">Troubleshooting</a></li>
@@ -446,7 +447,7 @@ <h2 id="X">X</h2>
   Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
     <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
     provided by <a href="https://readthedocs.org">Read the Docs</a>.
-   <jinja2.runtime.BlockReference object at 0x7efcc6866310> 
+   <jinja2.runtime.BlockReference object at 0x7fc02a16ca30> 
   <p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>
 
 

diff --git a/xpu/2.1.10+xpu/index.html b/xpu/2.1.10+xpu/index.html
@@ -57,6 +57,7 @@
 <li class="toctree-l1"><a class="reference internal" href="tutorials/introduction.html">Introduction</a></li>
 <li class="toctree-l1"><a class="reference internal" href="tutorials/features.html">Features</a></li>
 <li class="toctree-l1"><a class="reference internal" href="tutorials/llm.html">Large Language Models (LLM)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tutorials/performance.html">Performance</a></li>
 <li class="toctree-l1"><a class="reference internal" href="tutorials/technical_details.html">Technical Details</a></li>
 <li class="toctree-l1"><a class="reference internal" href="tutorials/releases.html">Releases</a></li>
 <li class="toctree-l1"><a class="reference internal" href="tutorials/performance_tuning/known_issues.html">Troubleshooting</a></li>
@@ -182,7 +183,7 @@ <h2>Support<a class="headerlink" href="#support" title="Permalink to this headin
   Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
     <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
     provided by <a href="https://readthedocs.org">Read the Docs</a>.
-   <jinja2.runtime.BlockReference object at 0x7efcc41784c0> 
+   <jinja2.runtime.BlockReference object at 0x7fc027a8c220> 
   <p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>
 
 

diff --git a/xpu/2.1.10+xpu/objects.inv b/xpu/2.1.10+xpu/objects.inv
diff --git a/xpu/2.1.10+xpu/py-modindex.html b/xpu/2.1.10+xpu/py-modindex.html
@@ -56,6 +56,7 @@
 <li class="toctree-l1"><a class="reference internal" href="tutorials/introduction.html">Introduction</a></li>
 <li class="toctree-l1"><a class="reference internal" href="tutorials/features.html">Features</a></li>
 <li class="toctree-l1"><a class="reference internal" href="tutorials/llm.html">Large Language Models (LLM)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tutorials/performance.html">Performance</a></li>
 <li class="toctree-l1"><a class="reference internal" href="tutorials/technical_details.html">Technical Details</a></li>
 <li class="toctree-l1"><a class="reference internal" href="tutorials/releases.html">Releases</a></li>
 <li class="toctree-l1"><a class="reference internal" href="tutorials/performance_tuning/known_issues.html">Troubleshooting</a></li>
@@ -150,7 +151,7 @@ <h1>Python Module Index</h1>
   Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
     <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
     provided by <a href="https://readthedocs.org">Read the Docs</a>.
-   <jinja2.runtime.BlockReference object at 0x7efcc43a0c40> 
+   <jinja2.runtime.BlockReference object at 0x7fc02a17ab80> 
   <p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>
 
 

diff --git a/xpu/2.1.10+xpu/search.html b/xpu/2.1.10+xpu/search.html
@@ -56,6 +56,7 @@
 <li class="toctree-l1"><a class="reference internal" href="tutorials/introduction.html">Introduction</a></li>
 <li class="toctree-l1"><a class="reference internal" href="tutorials/features.html">Features</a></li>
 <li class="toctree-l1"><a class="reference internal" href="tutorials/llm.html">Large Language Models (LLM)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tutorials/performance.html">Performance</a></li>
 <li class="toctree-l1"><a class="reference internal" href="tutorials/technical_details.html">Technical Details</a></li>
 <li class="toctree-l1"><a class="reference internal" href="tutorials/releases.html">Releases</a></li>
 <li class="toctree-l1"><a class="reference internal" href="tutorials/performance_tuning/known_issues.html">Troubleshooting</a></li>
@@ -133,7 +134,7 @@
   Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
     <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
     provided by <a href="https://readthedocs.org">Read the Docs</a>.
-   <jinja2.runtime.BlockReference object at 0x7efcc43a0520> 
+   <jinja2.runtime.BlockReference object at 0x7fc02a17a640> 
   <p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>
 
 

diff --git a/xpu/2.1.10+xpu/searchindex.js b/xpu/2.1.10+xpu/searchindex.js
diff --git a/xpu/2.1.10+xpu/tutorials/api_doc.html b/xpu/2.1.10+xpu/tutorials/api_doc.html
@@ -56,6 +56,7 @@
 <li class="toctree-l1"><a class="reference internal" href="introduction.html">Introduction</a></li>
 <li class="toctree-l1"><a class="reference internal" href="features.html">Features</a></li>
 <li class="toctree-l1"><a class="reference internal" href="llm.html">Large Language Models (LLM)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="performance.html">Performance</a></li>
 <li class="toctree-l1"><a class="reference internal" href="technical_details.html">Technical Details</a></li>
 <li class="toctree-l1"><a class="reference internal" href="releases.html">Releases</a></li>
 <li class="toctree-l1"><a class="reference internal" href="performance_tuning/known_issues.html">Troubleshooting</a></li>
@@ -1578,7 +1579,7 @@ <h3>Miscellaneous<a class="headerlink" href="#id1" title="Permalink to this head
   Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
     <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
     provided by <a href="https://readthedocs.org">Read the Docs</a>.
-   <jinja2.runtime.BlockReference object at 0x7efcc9154970> 
+   <jinja2.runtime.BlockReference object at 0x7fc02cb54eb0> 
   <p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>
 
 

diff --git a/xpu/2.1.10+xpu/tutorials/blogs_publications.html b/xpu/2.1.10+xpu/tutorials/blogs_publications.html
@@ -56,6 +56,7 @@
 <li class="toctree-l1"><a class="reference internal" href="introduction.html">Introduction</a></li>
 <li class="toctree-l1"><a class="reference internal" href="features.html">Features</a></li>
 <li class="toctree-l1"><a class="reference internal" href="llm.html">Large Language Models (LLM)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="performance.html">Performance</a></li>
 <li class="toctree-l1"><a class="reference internal" href="technical_details.html">Technical Details</a></li>
 <li class="toctree-l1"><a class="reference internal" href="releases.html">Releases</a></li>
 <li class="toctree-l1"><a class="reference internal" href="performance_tuning/known_issues.html">Troubleshooting</a></li>
@@ -167,7 +168,7 @@ <h1>Blogs &amp; Publications<a class="headerlink" href="#blogs-publications" tit
   Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
     <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
     provided by <a href="https://readthedocs.org">Read the Docs</a>.
-   <jinja2.runtime.BlockReference object at 0x7efcc40ffca0> 
+   <jinja2.runtime.BlockReference object at 0x7fc027413640> 
   <p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>
 
 

diff --git a/xpu/2.1.10+xpu/tutorials/cheat_sheet.html b/xpu/2.1.10+xpu/tutorials/cheat_sheet.html
@@ -56,6 +56,7 @@
 <li class="toctree-l1"><a class="reference internal" href="introduction.html">Introduction</a></li>
 <li class="toctree-l1"><a class="reference internal" href="features.html">Features</a></li>
 <li class="toctree-l1"><a class="reference internal" href="llm.html">Large Language Models (LLM)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="performance.html">Performance</a></li>
 <li class="toctree-l1"><a class="reference internal" href="technical_details.html">Technical Details</a></li>
 <li class="toctree-l1"><a class="reference internal" href="releases.html">Releases</a></li>
 <li class="toctree-l1"><a class="reference internal" href="performance_tuning/known_issues.html">Troubleshooting</a></li>
@@ -199,7 +200,7 @@ <h1>Cheat Sheet<a class="headerlink" href="#cheat-sheet" title="Permalink to thi
   Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
     <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
     provided by <a href="https://readthedocs.org">Read the Docs</a>.
-   <jinja2.runtime.BlockReference object at 0x7efcc40f70d0> 
+   <jinja2.runtime.BlockReference object at 0x7fc0278b25e0> 
   <p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>
 
 

diff --git a/xpu/2.1.10+xpu/tutorials/contribution.html b/xpu/2.1.10+xpu/tutorials/contribution.html
@@ -55,6 +55,7 @@
 <li class="toctree-l1"><a class="reference internal" href="introduction.html">Introduction</a></li>
 <li class="toctree-l1"><a class="reference internal" href="features.html">Features</a></li>
 <li class="toctree-l1"><a class="reference internal" href="llm.html">Large Language Models (LLM)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="performance.html">Performance</a></li>
 <li class="toctree-l1"><a class="reference internal" href="technical_details.html">Technical Details</a></li>
 <li class="toctree-l1"><a class="reference internal" href="releases.html">Releases</a></li>
 <li class="toctree-l1"><a class="reference internal" href="performance_tuning/known_issues.html">Troubleshooting</a></li>
@@ -274,7 +275,7 @@ <h4>Tips<a class="headerlink" href="#tips" title="Permalink to this heading">
   Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
     <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
     provided by <a href="https://readthedocs.org">Read the Docs</a>.
-   <jinja2.runtime.BlockReference object at 0x7efcc41199d0> 
+   <jinja2.runtime.BlockReference object at 0x7fc02744e8b0> 
   <p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>
 
 

diff --git a/xpu/2.1.10+xpu/tutorials/examples.html b/xpu/2.1.10+xpu/tutorials/examples.html
@@ -56,6 +56,7 @@
 <li class="toctree-l1"><a class="reference internal" href="introduction.html">Introduction</a></li>
 <li class="toctree-l1"><a class="reference internal" href="features.html">Features</a></li>
 <li class="toctree-l1"><a class="reference internal" href="llm.html">Large Language Models (LLM)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="performance.html">Performance</a></li>
 <li class="toctree-l1"><a class="reference internal" href="technical_details.html">Technical Details</a></li>
 <li class="toctree-l1"><a class="reference internal" href="releases.html">Releases</a></li>
 <li class="toctree-l1"><a class="reference internal" href="performance_tuning/known_issues.html">Troubleshooting</a></li>
@@ -1105,7 +1106,7 @@ <h2>Intel® AI Reference Models<a class="headerlink" href="#intel-ai-reference-m
   Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
     <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
     provided by <a href="https://readthedocs.org">Read the Docs</a>.
-   <jinja2.runtime.BlockReference object at 0x7efcc40d5100> 
+   <jinja2.runtime.BlockReference object at 0x7fc0278a3d00> 
   <p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>
 
 

diff --git a/xpu/2.1.10+xpu/tutorials/features.html b/xpu/2.1.10+xpu/tutorials/features.html
@@ -148,6 +148,7 @@
 </ul>
 </li>
 <li class="toctree-l1"><a class="reference internal" href="llm.html">Large Language Models (LLM)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="performance.html">Performance</a></li>
 <li class="toctree-l1"><a class="reference internal" href="technical_details.html">Technical Details</a></li>
 <li class="toctree-l1"><a class="reference internal" href="releases.html">Releases</a></li>
 <li class="toctree-l1"><a class="reference internal" href="performance_tuning/known_issues.html">Troubleshooting</a></li>
@@ -420,7 +421,7 @@ <h3>HyperTune (Experimental, <em>NEW feature in 1.13.0*</em>)<a class="headerlin
   Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
     <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
     provided by <a href="https://readthedocs.org">Read the Docs</a>.
-   <jinja2.runtime.BlockReference object at 0x7efcc409aee0> 
+   <jinja2.runtime.BlockReference object at 0x7fc0274135b0> 
   <p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>
 
 

diff --git a/xpu/2.1.10+xpu/tutorials/features/DDP.html b/xpu/2.1.10+xpu/tutorials/features/DDP.html
@@ -91,6 +91,7 @@
 </ul>
 </li>
 <li class="toctree-l1"><a class="reference internal" href="../llm.html">Large Language Models (LLM)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../performance.html">Performance</a></li>
 <li class="toctree-l1"><a class="reference internal" href="../technical_details.html">Technical Details</a></li>
 <li class="toctree-l1"><a class="reference internal" href="../releases.html">Releases</a></li>
 <li class="toctree-l1"><a class="reference internal" href="../performance_tuning/known_issues.html">Troubleshooting</a></li>
@@ -436,7 +437,7 @@ <h3>Usage of DDP scaling API<a class="headerlink" href="#usage-of-ddp-scaling-ap
   Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
     <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
     provided by <a href="https://readthedocs.org">Read the Docs</a>.
-   <jinja2.runtime.BlockReference object at 0x7efcc40f79d0> 
+   <jinja2.runtime.BlockReference object at 0x7fc029d0f7f0> 
   <p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>
 
 

diff --git a/xpu/2.1.10+xpu/tutorials/features/DLPack.html b/xpu/2.1.10+xpu/tutorials/features/DLPack.html
@@ -89,6 +89,7 @@
 </ul>
 </li>
 <li class="toctree-l1"><a class="reference internal" href="../llm.html">Large Language Models (LLM)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../performance.html">Performance</a></li>
 <li class="toctree-l1"><a class="reference internal" href="../technical_details.html">Technical Details</a></li>
 <li class="toctree-l1"><a class="reference internal" href="../releases.html">Releases</a></li>
 <li class="toctree-l1"><a class="reference internal" href="../performance_tuning/known_issues.html">Troubleshooting</a></li>
@@ -227,7 +228,7 @@ <h3>Example Case<a class="headerlink" href="#example-case" title="Permalink to t
   Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
     <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
     provided by <a href="https://readthedocs.org">Read the Docs</a>.
-   <jinja2.runtime.BlockReference object at 0x7efcc40f79d0> 
+   <jinja2.runtime.BlockReference object at 0x7fc0278e8a30> 
   <p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>