README.md

Adaptive Input Representations for Neural Language Modeling (Baevski and Auli, 2018)

Pre-trained models

Description	Parameters	Dataset	Model and Test set(s)
Adaptive Inputs (Baevski and Auli, 2018)	1026M	Google Billion Words	download (.tar.bz2)
Adaptive Inputs (Baevski and Auli, 2018)	247M	WikiText-103	download (.tar.bz2)

Example usage

See the language modeling README for instructions on reproducing results for WikiText-103 using the transformer_lm_wiki103 model architecture.

Citation

@inproceedings{
    baevski2018adaptive,
    title={Adaptive Input Representations for Neural Language Modeling},
    author={Alexei Baevski and Michael Auli},
    booktitle={International Conference on Learning Representations},
    year={2019},
    url={https://openreview.net/forum?id=ByxZX20qFQ},
}