Learn Transformer model (Attention is all you need, Google Brain, 2017) from implementation code written by @hyunwoongko in 2021.

• Min train loss = 2.7348
• Min validation loss = 3.2860
• Max blue score = 24.3375

• Attention is All You Need, 2017 - Google Brain
• The Illustrated Transformer - Jay Alammar
• Multi-Headed Attention implementation PyTorch
• Transformers from scratch blog
• 🤗 Hugging Face course
• How Transformers work in deep learning and NLP: an intuitive introduction
• Why multi-head self attention works: math, intuitions and 10+1 hidden insights

OpenAI GPT model using 🤗HuggingFace

BERT example from the Hugging Face transformer library