Recall and Confusion matrix, also the confusion matrix, of the WBF

[moahaimen]

I used the WBF for combining 3 yolov5 models, but now i need the final Precision and Recall and Confusion matrix, also the confusion matrix, of the WBF how can i do it,

[bshakya77]

@moahaimen, I am using YOLOv8n, YOLOv8s, YOLOv8m, YOLOv8l, YOLOv8x models with WBF for object detection in VisDrone2019 dataset.

Did you able to perform the evaluation for the ensemble model predictions?

Your feedback is highly appreciated.

Regards,
Bijay

[KAKAROT12419]

can you please tell me what format did weighted box fusion is giving output. Like its xmin, ymin , xmax, ymax or anything else.

[bshakya77]

can you please tell me what format did weighted box fusion is giving output. Like its xmin, ymin , xmax, ymax or anything else.

Yes, it gives output in xmin, ymin , xmax, ymax format. But, it is in de-normalized format.

Sample prediction result:

• bounding box coordinates
• conf scores
• class label

{'0000006_01659_d_0000004': [[404, 364, 216, 94], [452, 609, 156, 72], [435, 298, 196, 85], [463, 575, 145, 56], [446, 672, 160, 66], [489, 525, 123, 55], [551, 503, 106, 54], [442, 721, 164, 43], [481, 257, 182, 72], [509, 209, 164, 61], [541, 168, 137, 54], [364, 425, 209, 103], [684, 119, 33, 55], [712, 124, 37, 51], [743, 124, 21, 50]], '0000006_05168_d_0000013': [[594, 285, 62, 72], [743, 278, 21, 73], [83, 248, 131, 58], [679, 282, 58, 70], [103, 277, 126, 60], [137, 200, 123, 51], [195, 179, 110, 47], [528, 286, 59, 64], [0, 230, 31, 60], [123, 45, 16, 25], [115, 216, 131, 58]], '0000011_06595_d_0000010': [[746, 191, 18, 55], [697, 151, 18, 45], [513, 324, 36, 62], [496, 344, 75, 47], [340, 394, 60, 75], [255, 103, 20, 45], [533, 125, 18, 46]]}
{'0000006_01659_d_0000004': [[0.86604], [0.84769], [0.81754], [0.81647], [0.81016], [0.80896], [0.76495], [0.73147], [0.6655], [0.63703], [0.62914], [0.45945], [0.44228], [0.42985], [0.40013]], '0000006_05168_d_0000013': [[0.84664], [0.81553], [0.76947], [0.7288], [0.72482], [0.64824], [0.56348], [0.44876], [0.37089], [0.22063], [0.20727]], '0000011_06595_d_0000010': [[0.68253], [0.67364], [0.54689], [0.37154], [0.35386], [0.27636], [0.24566]]}
{'0000006_01659_d_0000004': [5, 3, 5, 3, 3, 3, 3, 3, 5, 5, 5, 5, 5, 5, 5], '0000006_05168_d_0000013': [3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 3], '0000011_06595_d_0000010': [6, 0, 1, 2, 2, 0, 0]}

I am trying to convert it in .csv format that is required for benchmark.
I am confused with the .csv annotation file used in benchmark file (ann). Is it annotation from test data for those images used in prediction?
Also, is it necessary that the prediction result (xmin, ymin , xmax, ymax) should be in normalized form?

Thank you,
Bijay

[KAKAROT12419]

Thanks for information Bijay. See i am not clear with benchmark file can you tell me something about benchmark file please.

[bshakya77]

Thanks for information Bijay. See i am not clear with benchmark file can you tell me something about benchmark file please.

I am referring to this file:
https://github.com/ZFTurbo/Weighted-Boxes-Fusion/blob/master/benchmark_oid/run_benchmark_oid.py

I see it takes the two .csv files: detection.csv and annotation.csv and calculates the mean average precision.

• I wanted to know if the (xmin, ymin , xmax, ymax) should be in normalized form?
• For annotation file, does the file contain the annotations of test image data from the dataset used (like those images used for predictions)?
• Have you calculated any other evaluation metrics like recall and accuracy for ensemble prediction?

Thank you,
Bijay

[tejasri19]

@moahaimen @bshakya77 Have you used any other ensemble techniques?
For YOLO models, images need not be normalised right for running inference

[bshakya77]

Hello,

@tejasri19 , so far I remember it does not require to be normalized for running inference. For evaluation, we need to normalized it in (XMin, XMax, YMin, YMax) format for both input files. I referred to this repo for evaluation.

https://github.com/ZFTurbo/Mean-Average-Precision-for-Boxes

Thank you.

Regards,
Bijay