The Training time increasses while increasing the number of GPU

[RJhuboo]

Hello,

Thank you for this work. I am trying to train RCAN from scratch on by dataset. I've noticed that my training time increases when I increase the number of GPU. For example when I used 4 GPU and 36 batchs I had a training time of 70s for 3200 images.

But when I used only 1 GPU for the same number of batchs, I had a training time of 31s for 3200 images.

I really can't explain why.

Do you have an idea ?

Thank you a lot

Rehan JHUBOO