Very Low GPU / CUDA usage (MOG2_CUDA is slow)

[sundeepgoel72]

Hi,

Even though I am using MOG2_CUDA , the GPU utilization remains poor (< 5%). How can I improve this and increase FPS throughput?

regards,
Sundeep

[Breakthrough]

This is a known limitation of the current GPU-accelerated version of DVR-Scan. See #12 for some background discussion on the topic.

Any suggestions or pull requests for improving performance are most welcome - will leave this issue open to focus on that area.

[Breakthrough]

I just did a test on a newer 3000 series GPU, and seem to get higher utilization closer to 20%. @sundeepgoel72 which GPU were you using this with? It may help to start collecting some kind of benchmarks from users.

[sundeepgoel72]

@sundeepgoel72 which GPU were you using this with

Older one : NVidia GTX 750 Ti

[dassio]

any update on this one, Should I buy a new version of GPU?
how much frame/s are you getting with a 3000 serials card?

[dassio]

i made a mistake the FAN utilization for GPU utilization: I should watch for the Volatile GPU-Util

I did a recent test with opencv-4.9.0 and GPU GTX 960, the GPU utilization is around 69%.

Thu Jul 18 18:11:17 2024
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 545.23.08              Driver Version: 545.23.08    CUDA Version: 12.3     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce GTX 960         On  | 00000000:26:00.0 Off |                  N/A |
| 43%   68C    P2              55W / 120W |    265MiB /  2048MiB |     69%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    0   N/A  N/A    426888      G   /usr/lib/xorg/Xorg                           47MiB |
|    0   N/A  N/A    426916      G   /usr/bin/gnome-shell                          4MiB |
|    0   N/A  N/A   1782189      C   /usr/bin/python3                            206MiB |
+---------------------------------------------------------------------------------------+

with two jobs running , I can reach 96% GPU utilization :

Thu Jul 18 18:13:33 2024
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 545.23.08              Driver Version: 545.23.08    CUDA Version: 12.3     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce GTX 960         On  | 00000000:26:00.0 Off |                  N/A |
| 40%   69C    P2              69W / 120W |    454MiB /  2048MiB |     95%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    0   N/A  N/A    426888      G   /usr/lib/xorg/Xorg                           47MiB |
|    0   N/A  N/A    426916      G   /usr/bin/gnome-shell                          4MiB |
|    0   N/A  N/A   1782189      C   /usr/bin/python3                            191MiB |
|    0   N/A  N/A   1789345      C   /usr/bin/python3                            204MiB |
+---------------------------------------------------------------------------------------+

and a test with RTX4090 with GPU utilization around 25%

hope the info will help