Intelligent Crowd Scene Analytics Engine

Introduction

Repository is modified from Mikel Brostrom code here: https://github.com/mikel-brostrom/Yolov5_DeepSort_Pytorch It contains a two-stage-tracker. The detections generated by YOLOv5, a family of object detection architectures and models pretrained on the COCO dataset, are passed to a Deep Sort algorithm which tracks the objects. It can track any object that your Yolov5 model was trained to detect.

Tutorials

• Yolov5 training on Custom Data (link to external repository) 
• DeepSort deep descriptor training (link to external repository) 
• Yolov5 deep_sort pytorch evaluation 

Before you run the tracker

1. Clone the repository recursively:

git clone --recurse-submodules https://github.com/adrielkuek/ITSS_2022.git

If you already cloned and forgot to use --recurse-submodules you can run git submodule update --init

2. Make sure that you fulfill all the requirements: Python 3.8 or later with all requirements.txt dependencies installed, including torch>=1.7. To install, run:

pip install -r requirements.txt

OR just installing the conda env yaml file using:

conda env create -f itss.yml

3. Download the model: crowdhuman_yolov5m.pt from link: (https://drive.google.com/file/d/1HoQHrIKwpc9ezim10hyWf7aQ0kyt0Upb/view?usp=sharing) place the model under dir yolov5/models/

Additional Notes

We will be using crowdhuman_yolov5m.pt as the best trade-off between runtime and accuracy on person detection. For DeepSORT appearance model, we will be using osnet_x1_0 for multi-scale invariance tracking performance. Open up CrowdAnalytics.ipynb either on local or colab and run through each cell.

Model parameters can be tweaked under configuration cell

augment = True          # Augmented inference
visual = False          # Visualisation function - Set this to false as default
conf_thres = 0.3        # Object confidence threshold
iou_thres = 0.5         # IOU Threshold for NMS
classes = 0             # Filter for class 0 - Person
agnostic_nms = True     # Class agnostic NMS
max_det = 1000          # Max number of detections per image
save_txt = True
save_vid = True

Tracking Output

Image frame and bounding box information

Output can be assigned to a list that stores the content of all the track boxes arising from every image frame

frame_idx + 1, id, bbox_left, bbox_top, bbox_w, bbox_h

Cite

If you find this project useful in your research, please consider cite:

@misc{NUS-ISS_ITSS2022,
    title={Intelligent Crowd Scene Analytics Engine},
    author={Adriel, Hao Zi, Pow Look, Konchok},
    url={https://github.com/adrielkuek/ITSS_2022.git},
    year={2022}
}