[Bug]: EMA doesnt work properly for batchsize>1. See attached paper

[pbrownrobo]

What happened?

EMA doesnt really work effectively when batchsize gets to be larger than one.
This is a well-known problem. OneTrainer is lacking the batch linked EMA scaling as mentioned in
https://arxiv.org/abs/2307.13813

What did you expect would happen?

EMA impedes training too much, for batchsize >2

Relevant log output

No response

Output of pip freeze

No response