Pretrained 3. ViT?

[vildesboe]

Hi! Thank you for publishing your code and for continuing to answer questions in this repo.
I am working on the survival task, and wonder if you pretrain the final ViT in any way, since you say you just "finetune" it for 20 epochs? Or is it finetuned from a random initialization?
Best regards