About Knowledge Distillation

[wnma3mz]

Hello, I am a beginner of knowledge distillation. Your code has helped me a lot. I have a question.

When using VGG, we need to use VGG_teacher.py to train to get the teacher model. Then we use VGG.py to distill S-DNN. Of course, the key part is SVD.

But why is there no pre-training teacher model in MobileNet.py and ResNext.py. This seems to be different from knowledge distillation, or I have neglected a certain part.

Looking forward to your reply. Thank you very much.