Transformers

Here are two notebooks of transformer implementation walk-troughs:

• Transformer encoder for Language Modeling implementation in PyTorch jupyter notebook

• Transformer for Language Translation in PyTorch jupyter notebook

And here are two notebooks of using pretrained GPT-2 model:

• Generating text with a pre-trained GPT2 in PyTorch jupyter notebook

• Fine-tuning GPT-2 on a jokes dataset in PyTorch jupyter notebook

I also wrote a blog post about fine-tuning pre-trained GPT2 model on a jokes dataset.