No accuracy boost with default settings

[agnonchik]

I don't observe any accuracy boost for dy_resnet18 compared to raw_resnet18 model on CIFAR10 and CIFAR100 datasets.
I used default hyper-parameters for the training and observed.
CIFAR10: dy 86.01%, raw 87.75%
CIFAR100: dy 54.59%, raw 56.52%.
All the experiments were bellow my expectations (>90% for CIFAR10, >70% for CIFAR100).
How to take advantage of dynamic convs?

[RongKaiWeskerMA]

The implementation looks fine according to the paper. But not sure if there are tricks to make the algorithm work. Or maybe it is not even working? Idk.