transformer-capacity

Theoretical estimates of transformer capacity

Name		Name	Last commit message	Last commit date
Latest commit History 15 Commits
.gitignore		.gitignore
2023-11-03-IT-iso-logit-directions-gpt2.ipynb		2023-11-03-IT-iso-logit-directions-gpt2.ipynb
2023-11-03-IT-iso-logit-directions-mistral.ipynb		2023-11-03-IT-iso-logit-directions-mistral.ipynb
2023-11-04-IT-decoding-logits.ipynb		2023-11-04-IT-decoding-logits.ipynb
LICENSE		LICENSE
README.md		README.md
babylm_dataset.py		babylm_dataset.py
bigrams.ipynb		bigrams.ipynb
compare-perplexity.ipynb		compare-perplexity.ipynb
data_and_tekoneizer.ipynb		data_and_tekoneizer.ipynb
data_preparatio_routines.py		data_preparatio_routines.py
entropy-temperature.ipynb		entropy-temperature.ipynb
gpt-2-outputs.ipynb		gpt-2-outputs.ipynb
hidden-state-outliers-gpt2.ipynb		hidden-state-outliers-gpt2.ipynb
mistral-outputs.ipynb		mistral-outputs.ipynb
n-grams.ipynb		n-grams.ipynb
nanogpt.py		nanogpt.py
new-setup-tests.ipynb		new-setup-tests.ipynb
poetry-entropy.ipynb		poetry-entropy.ipynb
thermodynamics.ipynb		thermodynamics.ipynb

Provide feedback