Question about teacher model

[Bgright]

First off, thank you for your time reasearching this field. It’s much appreciated and after finetuning my model on 2 datasets, I’m seeing postive results.

About the teacher model. To prevent forgetting, Should I keep changing the teacher model to the latest trained model? Or should I use the model which I used prior to starting any training?