Atlas 500

Model Conversion

The Ascend architecture does not run the models trained on traditional frameworks, such as TensorFlow or PyTorch. You have to convert them to something compatible, which is the case with the Da Vinci Offline Model.

In the current context we have the following setup and execution flow:

1. We have the YOLOv5 model trained (something like weights.pt) -- see section YOLOv5.
2. We convert weights.pt to weights.onnx -- see next section.
3. We then convert to weights.om via ATC, which will be deployed to a Python application (via pyACL) inside a docker image.
4. Finally, this image is uploaded to the Huawei Cloud and subsequently downloaded and run on the Atlas 500.

PyTorch & ONNX

In possession of the YOLOv5 repository and the trained model (something like weights.pt), we run the following command line to get the model in .onnx format

python yolov5/export.py --weights path/to/weights.pt --imgsz 384 --batch 1 --include onnx --opset 11 --simplify --device 0

In the end, if everything happens correctly, we should have a weights.onnx. Also, note that the opset (11) matters.

Compute Architecture for Neural Networks (CANN)

CANN is the hardware computing power base of the Ascend processors and ensures the execution of deep neural networks through its modules, such as Ascend CL, Graph Engine etc.

Links

• https://support.huawei.com/enterprise/br/doc/EDOC1100192470/7ebcb1b3/cann
• https://e.huawei.com/br/products/cloud-computing-dc/atlas/cann

Ascend Tool Compiler

ATC is a tool used to convert open-source models to offline models supported by Ascend processors.

In possession of the weights converted to .onnx (something like weights.onnx), we must run the command line below to obtain the desired Da Vinci model.

atc --input_shape="images:1,3,384,384" --check_report="network_analysis.report" --input_format="NCHW" --output="weights" --soc_version="Ascend310" --framework=5 --model="weights.onnx"

• input_shape represents the name (images) and shape (1, 3, 384, 384) of the input node of the network.
  • The order of the values follows input_format, which is batch_size, channels, height and width. The values here follow the parameters of the model trained and converted to .onnx.
• soc_version represents the final architecture to which the model will be converted -- Ascend310, the neural processing unit (UPN) present in the Atlas 500.
• framework represents the conversion output, which in this case is ONNX.
• model represents the name of the file to convert.
• output represents the name of the output file. In the end, it will be output_name.om, where .om stands for offline model.

Links

• https://support.huawei.com/enterprise/br/doc/EDOC1100192470/7d7ed5e8/atc-tool
• https://support.huaweicloud.com/intl/en-us/instg-cli-cann/atlascli_03_0007.html
• https://support.huaweicloud.com/intl/en-us/atctool-cann502alphaXinfer/atlasatc_16_0004.html

Model Deployment

PyACL

Python Ascend Computing Language (pyACL) is an API library in Python encapsulated using CPython based on the Ascend Common Language (ACL). Users can use Python to manage the execution and resources of Ascend processors.

Links

• https://toscode.gitee.com/tianyu__zhou/pyacl_samples/
• https://support.huaweicloud.com/intl/en-us/usermanual-cannApplicationDev330/atlaspython_01_0001.html
• https://support.huawei.com/enterprise/fr/doc/EDOC1100192513/7e88dc5e/pyacl-overview

Firmware

You must use version A500-3000-3010-firmware_20.3.0 (to be obtained here and followed the steps here) to successfully import and use the PyACL library.

Container

Creating the image

Creating the container with the inference scripts:

docker build -t hsc5g-a500 --build-arg NNRT_PKG=Ascend-cann-nnrt_20.3.0_linux-aarch64.run .

The relative Dockerfile is in this other repository, as are the scripts. Note the use of the NNRT package. This package contains the libs needed for inference with Ascend310, such as ACLlib.

To facilitate the deployment of the template in Atlas 500, we can upload the image to Huawei Cloud's Software Repository for Containers (SWR). See section Huawei Cloud - SWR.

The name of our Huawei Cloud image is: swr.la-south-2.myhuaweicloud.com/huawei-smart-cities/a500.

Container execution

After we download the containerized application image via docker pull, we can then run it:

docker run --device=/dev/davinci0 --device=/dev/davinci_manager --device=/dev/hisi_hdc --device /dev/devmm_svm -v /usr/bin/vi:/usr/local/bin/vi -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi -v /home/data/miniD/driver/lib64:/home/data/miniD/driver/lib64 -it swr.la-south-2.myhuaweicloud.com/huawei-smart-cities/a500:latest

where latest is the latest version of the image.

Links

• https://support.huaweicloud.com/intl/en-us/instg-container-image202/atlasdo_03_0007.html
• https://support.huaweicloud.com/intl/en-us/instg-container-image202/atlasdo_03_0008.html