Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates to merge latest changes for the RyzenAI-SW from the development branch (PR-117) #118

Merged
merged 48 commits into from
Aug 28, 2024
Merged
Changes from 1 commit
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
a16cf62
updates the read document for transformers and llama.cpp
cyndwith Aug 7, 2024
6474cdd
Merge branch 'main' into main
cyndwith Aug 7, 2024
7a26d29
update the ReadMe instruction across PyTorch, ONNX and Llama.cpp
cyndwith Aug 7, 2024
577e76b
Merge pull request #84 from savsrini/igpu-cpp
cyndwith Aug 7, 2024
3de4ba1
Merge pull request #85 from dwchenna/main
cyndwith Aug 7, 2024
f951935
Merge pull request #86 from VitisAI/main
savitha-srinivasan Aug 7, 2024
e7452f4
removing the --help instructions from llm onnx flow
cyndwith Aug 13, 2024
77e44c6
Update README.md
fanz-xlnx Aug 14, 2024
a59ea96
Updated Hello World Example for Ryzen AI SW 1.2 Release
vgodsoe Aug 16, 2024
4f88354
updates to onnx-benchmark to support STX and PHX
cyndwith Aug 27, 2024
f94c4c4
typo
giovanniguastiamd Aug 19, 2024
e9c088e
typo
giovanniguastiamd Aug 19, 2024
110589b
typo
giovanniguastiamd Aug 19, 2024
3b5427c
Merge pull request #89 from vigodsoe/helloworld-1-2-updates
cyndwith Aug 19, 2024
8b57cc5
PR
giovanniguastiamd Aug 20, 2024
3a97663
Update README.md
cyndwith Aug 21, 2024
a9ff650
updates to merge changes from dev to main branch
cyndwith Aug 27, 2024
0df9e3e
Merge branch 'main' into dev
cyndwith Aug 21, 2024
c2b63c9
merge changes from VitisAI/dev
cyndwith Aug 27, 2024
7657edf
fix to https://github.com/onnx/onnx/issues/6267
giovanniguastiamd Aug 22, 2024
fb64be1
fix for the build for windows
cyndwith Aug 27, 2024
c49c1a6
Refreshed Hello World example for 1.2 release, included new descripti…
vgodsoe Aug 23, 2024
c6bc8e5
Merge pull request #97 from vigodsoe/helloworld_1_2_updates
cyndwith Aug 23, 2024
455259e
adding clarificatoin to use Anaconda prompt for onnx-benchmark
cyndwith Aug 23, 2024
c62c8b5
Merge branch 'main' into main_ci_add
cyndwith Aug 23, 2024
2f6dd68
Merge pull request #96 from abalasa/main_ci_add
cyndwith Aug 23, 2024
3a8ea56
Merge branch 'dev' into unified_public
cyndwith Aug 23, 2024
35d3f6c
Merge pull request #95 from gguasti/unified_public
cyndwith Aug 23, 2024
4ce8d4e
Merge branch 'main' into dev
cyndwith Aug 23, 2024
efc9b6c
Merge branch 'main' of https://gitenterprise.xilinx.com/VitisAI/Ryzen…
cyndwith Aug 23, 2024
ce40f25
update the documentation for LLMs pytorch flow
cyndwith Aug 23, 2024
3b31e6a
Merge pull request #99 from VitisAI/llm_readme_updates
cyndwith Aug 24, 2024
95816f1
update cmakelist.txt to fixed compile issue OPENCV_DIR is add
Aug 26, 2024
9865e69
update multi-model demo
Aug 26, 2024
7264b50
update Opncv_DIR
Aug 26, 2024
8cb8856
update the readme
Aug 26, 2024
0a16c7c
update multi-model demo readme
Aug 26, 2024
630526b
Merge branch 'dev' into main
cyndwith Aug 26, 2024
60cc9cf
Merge branch 'dev' into main
cyndwith Aug 26, 2024
16f697f
Merge branch 'dev' into dev
cyndwith Aug 26, 2024
8ce4d48
Merge pull request #101 from VitisAI/main
cyndwith Aug 26, 2024
929663f
Merge branch 'dev' into dev
cyndwith Aug 26, 2024
2f4b57e
Merge pull request #100 from dachang/dev
cyndwith Aug 26, 2024
2a4fc84
Merge pull request #102 from VitisAI/dev
cyndwith Aug 26, 2024
7f51920
Merge pull request #1 from cyndwith/hot_fixes_v1.2.2
cyndwith Aug 27, 2024
c20f80b
updates to include dependencies for yolov8 tutorial
cyndwith Aug 27, 2024
b00783a
clean-up the CI config file
cyndwith Aug 28, 2024
a059a19
Merge branch 'main' into pr-117
cyndwith Aug 28, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
updates to include dependencies for yolov8 tutorial
cyndwith committed Aug 27, 2024
commit c20f80b3703ac3d0b4695d2775a74d851adf3872
265 changes: 0 additions & 265 deletions tutorial/yolov8/yolov8_cpp/README.md
Original file line number Diff line number Diff line change
@@ -5,39 +5,12 @@
</tr>
</table>

<<<<<<< HEAD
<<<<<<< HEAD
<<<<<<< HEAD
<<<<<<< HEAD
# Yolov8 on Ryzen AI


- Version: Ryzen AI Software v1.0
- Support: AMD Ryzen 7040U, 7040HS series mobile processors with Windows 11 OS.
- Last update: 4 Dec. 2023
=======
=======
>>>>>>> f83a0188 (Merge pull request #94 from VitisAI/dev)
=======
>>>>>>> 5be83bf9 (fix for https://github.com/onnx/onnx/issues/6267)
=======
>>>>>>> b02f64fa (Merge branch 'VitisAI:main' into main_ci_add)
# Yolov8 cpp flow on Ryzen AI


- Version: Ryzen AI Software v1.2
- Support: AMD Ryzen™ 7940HS, 7840HS, 7640HS, 7840U, 7640U, 8640U, 8640HS, 8645H, 8840U, 8840HS, 8845H, 8945H with Windows 11 OS.
- Last update: 29 Jul. 2024
<<<<<<< HEAD
<<<<<<< HEAD
<<<<<<< HEAD
>>>>>>> d78b7488 (Merge branch 'dev' into unified_public)
=======
>>>>>>> f83a0188 (Merge pull request #94 from VitisAI/dev)
=======
>>>>>>> 5be83bf9 (fix for https://github.com/onnx/onnx/issues/6267)
=======
>>>>>>> b02f64fa (Merge branch 'VitisAI:main' into main_ci_add)


## Table of Contents
@@ -48,25 +21,7 @@

[3 Installation](#3-installation)

<<<<<<< HEAD
<<<<<<< HEAD
<<<<<<< HEAD
<<<<<<< HEAD
[4 Quantization](#4-quantization)

[5 Implementation](#5-implementation)
=======
[4 Implementation](#4-implementation)
>>>>>>> d78b7488 (Merge branch 'dev' into unified_public)
=======
[4 Implementation](#4-implementation)
>>>>>>> f83a0188 (Merge pull request #94 from VitisAI/dev)
=======
[4 Implementation](#4-implementation)
>>>>>>> 5be83bf9 (fix for https://github.com/onnx/onnx/issues/6267)
=======
[4 Implementation](#4-implementation)
>>>>>>> b02f64fa (Merge branch 'VitisAI:main' into main_ci_add)

[License](#license)

@@ -79,29 +34,6 @@ In this Deep Learning(DL) tutorial, you will see how to deploy the Yolov8 detect

## 2 Prerequisites

<<<<<<< HEAD
<<<<<<< HEAD
<<<<<<< HEAD
<<<<<<< HEAD
- Linux server (GPU is preferred)
- AMD Ryzen AI Laptop with Windows 11 OS
- Visual Studio 2019 (with Desktop dev c++ & MSVC v142-vs2019 x64/x86 Spectre-mitigated libs)
- Anaconda or Miniconda
- Git
- openCV (version = 4.6.0)
- glog
- gflags
- cmake (version >= 3.26)
- python (version >= 3.9) (Recommended for python 3.9.13 64bit)
- IPU driver & IPU xclbin = 1.0 release
- voe package = 1.0 release
=======
=======
>>>>>>> f83a0188 (Merge pull request #94 from VitisAI/dev)
=======
>>>>>>> 5be83bf9 (fix for https://github.com/onnx/onnx/issues/6267)
=======
>>>>>>> b02f64fa (Merge branch 'VitisAI:main' into main_ci_add)
- AMD Ryzen AI Laptop with Windows 11 OS
- Visual Studio 2022 (with Desktop dev c++ & MSVC v143-vs2022 x64/x86 Spectre-mitigated libs)
- Anaconda or Miniconda
@@ -111,16 +43,6 @@ In this Deep Learning(DL) tutorial, you will see how to deploy the Yolov8 detect
- python (version = 3.10)
- NPU driver & NPU xclbin = 1.2 release
- voe package = 1.2 release
<<<<<<< HEAD
<<<<<<< HEAD
<<<<<<< HEAD
>>>>>>> d78b7488 (Merge branch 'dev' into unified_public)
=======
>>>>>>> f83a0188 (Merge pull request #94 from VitisAI/dev)
=======
>>>>>>> 5be83bf9 (fix for https://github.com/onnx/onnx/issues/6267)
=======
>>>>>>> b02f64fa (Merge branch 'VitisAI:main' into main_ci_add)

## 3 Installation

@@ -132,23 +54,7 @@ There are some more libraries you need to install for the Yolov8 inference.

#### Cmake

<<<<<<< HEAD
<<<<<<< HEAD
<<<<<<< HEAD
<<<<<<< HEAD
```Conda Prompt
=======
```Anaconda Prompt
>>>>>>> d78b7488 (Merge branch 'dev' into unified_public)
=======
```Anaconda Prompt
>>>>>>> f83a0188 (Merge pull request #94 from VitisAI/dev)
=======
```Anaconda Prompt
>>>>>>> 5be83bf9 (fix for https://github.com/onnx/onnx/issues/6267)
=======
```Anaconda Prompt
>>>>>>> b02f64fa (Merge branch 'VitisAI:main' into main_ci_add)
# pip install cmake
```

@@ -168,30 +74,6 @@ Successfully installed cmake-3.27.4.1

It is recommended to build OpenCV form source code and use static build. [Git](https://git-scm.com/download/win) is required to clone the repository.

<<<<<<< HEAD
<<<<<<< HEAD
<<<<<<< HEAD
<<<<<<< HEAD
Start a `Git Bash`. In the Git Bash, clone the repository

```Git Bash
# git clone https://github.com/opencv/opencv.git -b 4.6.0
```

Switch back to the `Conda Prompt`, and compile the OpenCV source code with cmake.

```Conda Prompt
# cd opencv
# mkdir mybuild
# cd mybuild
# cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DBUILD_SHARED_LIBS=OFF -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_CONFIGURATION_TYPES=Release -A x64 -T host=x64 -G "Visual Studio 16 2019" '-DCMAKE_INSTALL_PREFIX=C:\Program Files\opencv' '-DCMAKE_PREFIX_PATH=.\opencv' -DCMAKE_BUILD_TYPE=Release -DBUILD_opencv_python2=OFF -DBUILD_opencv_python3=OFF -DBUILD_WITH_STATIC_CRT=OFF -B build -S ../
=======
=======
>>>>>>> f83a0188 (Merge pull request #94 from VitisAI/dev)
=======
>>>>>>> 5be83bf9 (fix for https://github.com/onnx/onnx/issues/6267)
=======
>>>>>>> b02f64fa (Merge branch 'VitisAI:main' into main_ci_add)
Start a `Anaconda Prompt`. In your workspace, clone the repository

```Anaconda Prompt
@@ -205,124 +87,11 @@ Then compile the OpenCV source code with cmake.
# mkdir mybuild
# cd mybuild
# cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DBUILD_SHARED_LIBS=OFF -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_CONFIGURATION_TYPES=Release -A x64 -T host=x64 -G "Visual Studio 17 2022" '-DCMAKE_INSTALL_PREFIX=C:\Program Files\opencv' '-DCMAKE_PREFIX_PATH=.\opencv' -DCMAKE_BUILD_TYPE=Release -DBUILD_opencv_python2=OFF -DBUILD_opencv_python3=OFF -DBUILD_WITH_STATIC_CRT=OFF -B build -S ../
<<<<<<< HEAD
<<<<<<< HEAD
<<<<<<< HEAD
>>>>>>> d78b7488 (Merge branch 'dev' into unified_public)
=======
>>>>>>> f83a0188 (Merge pull request #94 from VitisAI/dev)
=======
>>>>>>> 5be83bf9 (fix for https://github.com/onnx/onnx/issues/6267)
=======
>>>>>>> b02f64fa (Merge branch 'VitisAI:main' into main_ci_add)
# cmake --build build --config Release
# cmake --install build --config Release
# cd ../..
```

<<<<<<< HEAD
<<<<<<< HEAD
<<<<<<< HEAD
<<<<<<< HEAD
All the dependencies on the Ryzen AI laptop are installed completely. User could run a end to end Yolov8 deplomyment progress with the following ***Section 4***, which will start from the FP32 Yolov8 model. The whole progress will last for several hours or one day depending on the hardware computing ability.

Alternatively, user who wants a quick benchmark could skip ***Section 4*** and start from ***Section 5*** with pre-quantized model.

## 4 Quantization

In this section, we will leverage the Ryzen AI docker container on Linux GPU server for a quantized awared training(QAT).

Please follow the instrucion [here](https://ryzenai.docs.amd.com/en/latest/alternate_quantization_setup.html#install-pt-tf) to build your docker container or pull prebuild docker from docker hub.

This tutorial will take GPU docker as a reference.

### Build Vitis AI GPU Docker

```Bash
$ cd <ryzen-ai-gpudockerfiles>
$ ./docker_build.sh -t gpu -f pytorch
```

### Prepare Coco Dataset

Download the COCO dataset from https://cocodataset.org/#download following the instruction and make sure the dataset structure is restored as below.
Please also update variable "DATA_PATH" in "coco.yaml" to point to the correct location.

```markdown
+ datasets/
+ coco/
+ labels/
+ annotations/
+ images/
+ test-dev2017.txt
+ train2017.txt
+ val2017.txt
```

### Quantization
Clone the RyzenAI-SW repository.
```Bash
$ git clone https://github.com/amd/RyzenAI-SW.git
```

Start a Docker container using the image.

```Bash
$ docker run -it --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --device=/dev/kfd --device=/dev/dri --group-add video --ipc=host --shm-size 8G xilinx/vitis-ai-pytorch-gpu:<Your-Image-Tag>
```

You can also pass the ***-v*** argument to mount any data directories from the host onto the container.

Then, setup the environment with following commands.

```Bash
$ cd RyzenAI-SW/tutorial/yolov8_e2e
$ sudo bash env_setup.sh
$ cd code
$ python3 setup.py develop
```

User could use the ***run_test.sh*** script to validate the float point model first before the quantization.

```Bash
$ bash run_test.sh
```

Then Quantize the model with following script.

```Bash
$ bash run_ptq.sh
```

Then quantize the model with QAT technique.

```Bash
$ bash run_qat.sh
```

Copy the quantized model to Ryzen AI laptop for the following deployment.

## 5 Implementation

### Compilation

If the ***section 4*** is skiped, please start a `Git Bash`. In the Git Bash, clone the repository

```Git Bash
# git clone https://github.com/amd/RyzenAI-SW.git
```

Switch back to the `Conda Prompt`, and compile the Yolov8 source code.

```Conda Prompt
# cd RyzenAI-SW/tutorial/yolov8_e2e/implement
=======
=======
>>>>>>> f83a0188 (Merge pull request #94 from VitisAI/dev)
=======
>>>>>>> 5be83bf9 (fix for https://github.com/onnx/onnx/issues/6267)
=======
>>>>>>> b02f64fa (Merge branch 'VitisAI:main' into main_ci_add)
All the dependencies on the Ryzen AI laptop are installed completely. User could run a Yolov8 deplomyment progress in cpp with int8 Yolov8 pre-quantized model in the following ***Section 4***,

## 4 Implementation
@@ -339,50 +108,16 @@ Then compile the Yolov8 source code.

```Anaconda Prompt
# cd RyzenAI-SW/tutorial/yolov8_cpp/implement
<<<<<<< HEAD
<<<<<<< HEAD
<<<<<<< HEAD
>>>>>>> d78b7488 (Merge branch 'dev' into unified_public)
=======
>>>>>>> f83a0188 (Merge pull request #94 from VitisAI/dev)
=======
>>>>>>> 5be83bf9 (fix for https://github.com/onnx/onnx/issues/6267)
=======
>>>>>>> b02f64fa (Merge branch 'VitisAI:main' into main_ci_add)
# build.bat
```

The output will be generated as below.

```
......
<<<<<<< HEAD
<<<<<<< HEAD
<<<<<<< HEAD
<<<<<<< HEAD
-- Installing: C:/Users/ibane/Desktop/voe-win_amd64-with_xcompiler_on-c07e419-latest/bin/camera_yolov8.exe
-- Installing: C:/Users/ibane/Desktop/voe-win_amd64-with_xcompiler_on-c07e419-latest/bin/camera_yolov8_nx1x4.exe
-- Installing: C:/Users/ibane/Desktop/voe-win_amd64-with_xcompiler_on-c07e419-latest/bin/test_jpeg_yolov8.exe
=======
-- Installing: C:/Users/fanz/Documents/Workspace/RyzenAI-SW/tutorial/yolov8_cpp/bin/camera_yolov8.exe
-- Installing: C:/Users/fanz/Documents/Workspace/RyzenAI-SW/tutorial/yolov8_cpp/bin/camera_yolov8_nx1x4.exe
-- Installing: C:/Users/fanz/Documents/Workspace/RyzenAI-SW/tutorial/yolov8_cpp/bin/test_jpeg_yolov8.exe
>>>>>>> d78b7488 (Merge branch 'dev' into unified_public)
=======
-- Installing: C:/Users/fanz/Documents/Workspace/RyzenAI-SW/tutorial/yolov8_cpp/bin/camera_yolov8.exe
-- Installing: C:/Users/fanz/Documents/Workspace/RyzenAI-SW/tutorial/yolov8_cpp/bin/camera_yolov8_nx1x4.exe
-- Installing: C:/Users/fanz/Documents/Workspace/RyzenAI-SW/tutorial/yolov8_cpp/bin/test_jpeg_yolov8.exe
>>>>>>> f83a0188 (Merge pull request #94 from VitisAI/dev)
=======
-- Installing: C:/Users/fanz/Documents/Workspace/RyzenAI-SW/tutorial/yolov8_cpp/bin/camera_yolov8.exe
-- Installing: C:/Users/fanz/Documents/Workspace/RyzenAI-SW/tutorial/yolov8_cpp/bin/camera_yolov8_nx1x4.exe
-- Installing: C:/Users/fanz/Documents/Workspace/RyzenAI-SW/tutorial/yolov8_cpp/bin/test_jpeg_yolov8.exe
>>>>>>> 5be83bf9 (fix for https://github.com/onnx/onnx/issues/6267)
=======
-- Installing: C:/Users/fanz/Documents/Workspace/RyzenAI-SW/tutorial/yolov8_cpp/bin/camera_yolov8.exe
-- Installing: C:/Users/fanz/Documents/Workspace/RyzenAI-SW/tutorial/yolov8_cpp/bin/camera_yolov8_nx1x4.exe
-- Installing: C:/Users/fanz/Documents/Workspace/RyzenAI-SW/tutorial/yolov8_cpp/bin/test_jpeg_yolov8.exe
>>>>>>> b02f64fa (Merge branch 'VitisAI:main' into main_ci_add)
```

### Run with Image
3 changes: 3 additions & 0 deletions tutorial/yolov8/yolov8_cpp/bin/DirectML.dll
Git LFS file not shown
3 changes: 3 additions & 0 deletions tutorial/yolov8/yolov8_cpp/bin/gflags.dll
Git LFS file not shown
3 changes: 3 additions & 0 deletions tutorial/yolov8/yolov8_cpp/bin/gflags_nothreads.dll
Git LFS file not shown
3 changes: 3 additions & 0 deletions tutorial/yolov8/yolov8_cpp/bin/glog.dll
Git LFS file not shown
3 changes: 3 additions & 0 deletions tutorial/yolov8/yolov8_cpp/bin/onnxruntime.dll
Git LFS file not shown
Git LFS file not shown
Git LFS file not shown
3 changes: 3 additions & 0 deletions tutorial/yolov8/yolov8_cpp/bin/onnxruntime_vitisai_ep.dll
Git LFS file not shown
Loading