This is probably going to improve things for end-user but when we update our Anki and optimise, this should result in worse log-loss in which Anki will just retain the previous parameters instead of updating them. Wdyt about that?

This is probably going to improve things for end-user but when we update our Anki and optimise, this should result in worse log-loss in which Anki will just retain the previous parameters instead of updating them. Wdyt about that?

Maybe we need apply the weights in the evaluation, too.

I've said this before - if we apply weights during both evaluation and training, then loss values will effectively become arbitrary since they depend on the weighting function.