Tutorial in Chinese can be found in mofanpy.com.

This repo includes many simple implementations of models in Neural Language Processing (NLP).

All code implementations in this tutorial are organized as following:

1. Search Engine

• TF-IDF numpy / TF-IDF skearn

2. Understand Word

• Continuous Bag of Words (CBOW)
• Skip-Gram

3. Understand Sentence

• seq2seq
• CNN language model

4. All about Attention

• seq2seq with attention
• Transformer

5. Pretrained Models

• ELMo
• GPT
• BERT or my new attempt Bert with next mask

TF-IDF visualization

word2vec visualization

seq2seq attention visualization

transformer encoder-decoder attention visualization

bert next masked attention visualization