Setup • FPGA demo • Steps for FPGA inference • Results • Pretrained models • Contact
Covid4HPC is a Deep Learning application that uses Neural Networks to detect Covid-19 disease from X-Ray images. Also, the application is accelerated through Cloud FPGAs (specifically Xilinx Alveo U50) and achieves significant performance and accuracy as well. This project is not intended for self-diagnosis and anyone should seek help from their local health authorities if needed.
- Input data: 224x224 chest X-ray images
- Dataset: Kaggle COVID-19 Radiography Database [link]
- Classes: "COVID-19", "Normal", "Viral Pneumonia"
- Accuracy: 97%
- Inference Speed (FPGA): 3700 FPS
- An Ubuntu host PC (16.04 or 18.04)
- Setup Alveo U50 FPGA Card if you haven't already. Install XRT and Alveo target platform files. [Follow instructions]
- Install Docker and ensure your linux user is in the docker group:
sudo apt install docker -y
sudo groupadd docker
sudo usermod -aG docker $USER
newgrp docker- Download our github repo (~70MB in size)
git clone https://github.com/dimdano/Covid4HPC.git- Pull and run Vitis AI prebuilt docker (release 1.3.411 recommended) - Vitis AI.
chmod +x docker_run.sh
./docker_run.sh xilinx/vitis-ai:1.3.411- Install Xilinx DPU IP for the Alveo U50. While inside docker run:
wget https://www.xilinx.com/bin/public/openDownload?filename=alveo_xclbin-1.3.1.tar.gz -O alveo_xclbin-1.3.1.tar.gz
tar xfz alveo_xclbin-1.3.1.tar.gz
sudo cp alveo_xclbin-1.3.1/U50/6E300M/* /usr/lib- You can head to FPGA-demo to test the ready-to-run application. For the other tutorials you will need to download COVID-19 Radiography dataset from here. Then place the 3 class folders (Covid, Normal, Viral Pneumonia) inside the dataset folder of the repo.
Follow the next steps to run the Vitis-AI tools for FPGA deployment and inference of the model. The tutorial runs for CustomCNN model by default. If you would like to use the DenseNetX model run source ./0_setenv.sh DenseNetX instead of source ./0_setenv.sh.
- First run the script to set variables and activate vitis-ai-tensorflow:
source ./0_setenv.sh- Convert the model to TF Frozen Graph by running the following command:
source ./1_keras2tf.sh- Evaluate the prediction accuracy on the 32-bit model.
source ./2_eval_frozen.sh- Quantize from 32-bit floating point to 8-bit fixed point with calibration images:
source ./3_quant.sh- Evaluate the prediction accuracy of the quantized 8-bit model:
source ./4_eval_quant.sh- Compile the quantized model and generate the xmodel file for the target FPGA (Alveo U50):
source ./5_compile.sh - Prepare the application folder and generate the test images:
source ./6_make_target.sh- Go to
build/target/folder and run application with:
/usr/bin/python3 run_inference.py -m model_dir/fpga_model.xmodel -t 6Note* The DPU runs at 300MhZ. If you experience issues with the FPGA becoming unstable, it might need reset. You can scale down the frequency to overcome these issues. For example run the following to reduce to 250Mhz:
/opt/xilinx/xrt/bin/xbutil program -p /usr/lib/dpu.xclbin
/opt/xilinx/xrt/bin/xbutil clock -d0 -g 83If you want to test the ready-to-run application first conda activate vitis ai tensorflow package:
conda activate vitis-ai-tensorflowThen head to FPGA_demo/ folder and run the following command to test the CustomCNN model on FPGA:
/usr/bin/python3 run_inference.py -m model_dir/CustomCNN.xmodel -t 6The Frames/Sec (FPS) and Accuracy achieved will be shown. Also, a .txt file will be generated with the predicted images. You can select the model to infer with -m option or the number of threads with -t option. (If you experience issues with FPGA see Note* ) Output:
-----------------------------------------
Running on Alveo U50
-----------------------------------------
Command line options:
--image_dir : images
--threads : 6
--model : model_dir/fpga_model.xmodel
-----------------------------------------
Number of images: 727
FPS=3778.39, total frames = 727 , time=0.2016 seconds
Correct: 704 Wrong: 23 Accuracy:0.9684A jupyter notebook cpu-evaluation.ipynb/ is provided to evaluate the model on several classification metrics in SW. Also, you can visualize the regions of attention of the X-ray images. First source Vitis AI environment:
source ./0_setenv.sh For the visualization package only, you will need to install a higher TF-version and tf_keras_vis package by running the following command:
pip install tensorflow==2.0.2 tf_keras_vis Then, while inside the docker container you can run jupyter (port 8888 enabled in host).
jupyter notebook --no-browser --port=8888 --allow-rootNext, copy the URL shown afterwards in a host browser to run jupyter.
The results here are reported for the "CustomCNN" model but you can easily change the model through the notebook provided.
Loss: 0.10907891045799774
Accuracy: 0.97111416
========================================================
precision recall f1-score support
COVID-19 1.00 0.95 0.97 55
NORMAL 0.98 0.96 0.97 335
Viral Pneumonia 0.96 0.98 0.97 337
accuracy 0.97 727
macro avg 0.98 0.96 0.97 727
weighted avg 0.97 0.97 0.97 727
========================================================
We compared the throughput of the model in FPGA with CPU and GPU using the default settings for an inference in keras.
| Architecture | System | Frames/second |
|---|---|---|
| CPU | Intel Xeon Silver 4210 (20-thread) | 204 |
| GPU | Nvidia V100 | 1157 |
| FPGA | Xilinx Alveo U50 | 3606 |
Another way to demonstrate the results in the qualitative comparison, was to generate saliency maps for our model’s predictions using tf-keras-vis package. These visualizations are very important because they give us an insight on the model's classifier capability as well as validate the regions of attention.
This software uses the following open source packages:
Dimitrios Danopoulos: [email protected]