Fruit Counting Engine

Motivation:

Building a camera module mounted on a UAV that collects data on images of dragon fruit and gives an estimate of dragon fruit production

Hardware

• Jetson AGX Xavier (32GB)
• ZED 2 camera
• Drone
• RTK

Software

• Jetpack 4.6 (rev.3)
• ZED SDK (ZED SDK 3.6.5 - have some problems with version 3.7)

Tasks

• Model dragon fruit detection (SSD-Mobilenet-v2/Yolov5)
  • Train test
  • Dataset
  • Evaluate model

Phase 1

• Implement model in Jetson AGX and ZED 2
  • Run TenserRT Engine in ZED 2 Video stream
    • Build and Run
    • Fixing crash (fixed bug => ZED SDK 3.6.5)
  • Bounding Box Tracker
    • MO Tracker (testing)
  • Counting Algorithm
  • Test with dragon fruit model and data
  • Simple GUI
    • Control start/stop camera
    • Counting control

Usage

1. Requierments

• ZED SDK (ZED SDK 3.6.5)
• OpenCV 4.2
• CUDA 10.2
• TensorRT 8
• Qt 5.12

2. Train model

2.1. YOLOv5

• Install YoLov5 requiements

pip install -r Model/Yolo/yolov5/requirements.txt

# Install ONNX lib
pip install onnx

• Use this file for split data to train, test and vali folder with yolo data format (only support VOC data format).

• Change path of dataset and class name in this file

• Train model by using this file

• Export model to ONNX format by using this file

2.2. SSD-MobileNet

• Install requirement

cd Model/ssd/
pip install -r requirements.txt

• Using this file for convert data to VOC format

• Download pre-train net

wget -P models https://storage.googleapis.com/models-hao/mobilenet-v1-ssd-mp-0_675.pth

• Train model by using this file

• Evaluation model by using this file

• Convert to ONNX by using this file

3. Convert model to TensorRT Engine

# gennerate .wst file
cd Models/Yolo/yolov5
python gen_wts.py <path to .pt file>

cd TensorRT-Engine/yolov5/

# Copy your trained model to here

mkdir build & cd build
cmake ..
make

# Copy .wts file generated to build folder

# build engine
./yolov5_zed -s best.wts test-11.engine s

4. Using GUI

# Copy .engine file to Engine/Data/model
# Build GUI
cd Engine
mkdir build
cd build
cmake ..
make

# Run
./EngineGUI

Report (Pending)

Ref

https://blog.paperspace.com/train-yolov5-custom-data/

https://github.com/jkjung-avt/tensorrt_demos