compact-convolutional-transformer

Implementation of Compact Convolutional Transformer (CCT) from Escaping the Big Data Paradigm with Compact Transformers
https://arxiv.org/abs/2104.05704

Some code borrowed from the following tutorials:
https://medium.com/pytorch/training-compact-transformers-from-scratch-in-30-minutes-with-pytorch-ff5c21668ed5 https://medium.com/the-dl/transformers-from-scratch-in-pytorch-8777e346ca51 https://blog.floydhub.com/the-transformer-in-pytorch/

Citation

@article{hassani2021escaping,
	title        = {Escaping the Big Data Paradigm with Compact Transformers},
	author       = {Ali Hassani and Steven Walton and Nikhil Shah and Abulikemu Abuduweili and Jiachen Li and Humphrey Shi},
	year         = 2021,
	url          = {https://arxiv.org/abs/2104.05704},
	eprint       = {2104.05704},
	archiveprefix = {arXiv},
	primaryclass = {cs.CV}
}