This project is intended for training a language-only transformer using PyTorch. It was started on New Year's Eve 2022.
- Character-level tokenisation and general batching complete (7/1/23).
- Minimal model training! (Linear model on char-tokenised data). (8/1/23).
- Complete transformer model with positional embeddings (2/3/23).
- Minimal single-device training loop on wikitext (13/3/23).