+
+ +
+

Performance

+
+

Overview

+

This page shows performance boost with Intel® Extension for PyTorch* on several popular topologies.

+
+
+

Performance Data for Intel® AI Data Center Products

+

Find the latest performance data for Intel® Data Center Max 1550 GPU, including detailed hardware and software configurations.

+
+
+

LLM Performance

+

We benchmarked GPT-J 6B, LLaMA2 7B, 13B, OPT 6.7B, Bloom-7B with test input token length set to 1024. The datatype is FP16 for all the models.

+

Single Tile

+

Single Card

+

Two Card

+

Four Card

+
+

Configuration

+
+

Software Version

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
SoftwareVersion
PyTorchv2.1
Intel® Extension for PyTorch*v2.1.10+xpu
Intel® oneAPI Base Toolkit2024.0
Torch-CCL2.1.100
GPU Driver736.25
Transformersv4.31.0
DeepSpeedcommit 4fc181b0
Intel® Extension for DeepSpeed*commit ec33277
+
+

Hardware Configuration

+

CPU Configuration:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
CPUIntel(R) Xeon(R) Platinum 8480+ CPU
Number of nodes1
Number of sockets2
Cores/Socket56
Threads/Core2
uCode0x2b0004b1
Hyper-ThreadingON
TurboBoostON
BIOS versionSE5C7411.86B.9525.D25.2304190630
Number of DDR Memory slots16
Capacity of DDR memory per slot64GB
DDR frequency4800
Total Memory/Node (DDR+DCPMM)1024GB
Host OSUbuntu 22.04.3 LTS
Host Kernel5.17.0-1020-oem
Spectre-Meltdown MitigationMitigated

Single tile of 4X PVC OAM Configuration:

+ + + + + + + + + + + + + + + + + + + + + + + + + +
GPUIntel(R) Data Center Max 1550 GPU
IFWIPVC.PS.B4.P.Si.2023.WW42.3_25MHzi_Quad_DAMeni_OAM600W_IFRv2332i_PSCnull_IFWI.bin
ECCON
AMC SWAMC FW 6.2
PrecisionFP16
+
+
+
+ + +
+