NLP-Models-Tensorflow, Gathers machine learning and tensorflow deep learning models for NLP problems, code simplify inside Jupyter Notebooks 100%.
Table of contents
- Text classification
- Chatbot
- Neural Machine Translation
- Embedded
- Entity-Tagging
- POS-Tagging
- Dependency-Parser
- Question-Answers
- Abstractive Summarization
- Extractive Summarization
- Stemming
- Generator
- Topic Generator
- Language detection
- OCR (optical character recognition)
- Speech to Text
- Text to Speech
- Old-to-Young Vocoder
- Text Similarity
- Text Augmentation
- Miscellaneous
- Attention
Objective
Original implementations are quite complex and not really beginner friendly. So I tried to simplify most of it. Also, there are tons of not-yet release papers implementation. So feel free to use it for your own research!
I will attached github repositories for models that I not implemented from scratch, basically I copy, paste and fix those code for deprecated issues.
Contents
Text classification
- Basic cell RNN
- Bidirectional RNN
- LSTM cell RNN
- GRU cell RNN
- LSTM RNN + Conv2D
- K-max Conv1d
- LSTM RNN + Conv1D + Highway
- LSTM RNN with Attention
- Neural Turing Machine
- Bidirectional Transformers
- Dynamic Memory Network
- Residual Network using Atrous CNN + Bahdanau Attention
- Transformer-XL
Complete list (71 notebooks)
- Basic cell RNN
- Basic cell RNN + Hinge
- Basic cell RNN + Huber
- Basic cell Bidirectional RNN
- Basic cell Bidirectional RNN + Hinge
- Basic cell Bidirectional RNN + Huber
- LSTM cell RNN
- LSTM cell RNN + Hinge
- LSTM cell RNN + Huber
- LSTM cell Bidirectional RNN
- LSTM cell Bidirectional RNN + Huber
- LSTM cell RNN + Dropout + L2
- GRU cell RNN
- GRU cell RNN + Hinge
- GRU cell RNN + Huber
- GRU cell Bidirectional RNN
- GRU cell Bidirectional RNN + Hinge
- GRU cell Bidirectional RNN + Huber
- LSTM RNN + Conv2D
- K-max Conv1d
- LSTM RNN + Conv1D + Highway
- LSTM RNN + Basic Attention
- LSTM Dilated RNN
- Layer-Norm LSTM cell RNN
- Only Attention Neural Network
- Multihead-Attention Neural Network
- Neural Turing Machine
- LSTM Seq2Seq
- LSTM Seq2Seq + Luong Attention
- LSTM Seq2Seq + Bahdanau Attention
- LSTM Seq2Seq + Beam Decoder
- LSTM Bidirectional Seq2Seq
- Pointer Net
- LSTM cell RNN + Bahdanau Attention
- LSTM cell RNN + Luong Attention
- LSTM cell RNN + Stack Bahdanau Luong Attention
- LSTM cell Bidirectional RNN + backward Bahdanau + forward Luong
- Bytenet
- Fast-slow LSTM
- Siamese Network
- LSTM Seq2Seq + tf.estimator
- Capsule layers + RNN LSTM
- Capsule layers + LSTM Seq2Seq
- Capsule layers + LSTM Bidirectional Seq2Seq
- Nested LSTM
- LSTM Seq2Seq + Highway
- Triplet loss + LSTM
- DNC (Differentiable Neural Computer)
- ConvLSTM
- Temporal Convd Net
- Batch-all Triplet-loss + LSTM
- Fast-text
- Gated Convolution Network
- Simple Recurrent Unit
- LSTM Hierarchical Attention Network
- Bidirectional Transformers
- Dynamic Memory Network
- Entity Network
- End-to-End Memory Network
- BOW-Chars Deep sparse Network
- Residual Network using Atrous CNN
- Residual Network using Atrous CNN + Bahdanau Attention
- Deep pyramid CNN
- Transformer-XL
- GPT-2
- Quasi-RNN
- Tacotron
- Slice GRU
- Slice GRU + Bahdanau
- Wavenet
- Transfer learning BERT base
Chatbot
- Seq2Seq-manual
- Seq2Seq-API Greedy
- Bidirectional Seq2Seq-manual
- Bidirectional Seq2Seq-API Greedy
- Bidirectional Seq2Seq-manual + backward Bahdanau + forward Luong
- Bidirectional Seq2Seq-API + backward Bahdanau + forward Luong + Stack Bahdanau Luong Attention + Beam Decoder
- Bytenet
- Capsule layers + LSTM Seq2Seq-API + Luong Attention + Beam Decoder
- End-to-End Memory Network
- Attention is All you need
- Transformer-XL + LSTM
- GPT-2 + LSTM
- Tacotron + Beam decoder
Complete list (54 notebooks)
- Basic cell Seq2Seq-manual
- LSTM Seq2Seq-manual
- GRU Seq2Seq-manual
- Basic cell Seq2Seq-API Greedy
- LSTM Seq2Seq-API Greedy
- GRU Seq2Seq-API Greedy
- Basic cell Bidirectional Seq2Seq-manual
- LSTM Bidirectional Seq2Seq-manual
- GRU Bidirectional Seq2Seq-manual
- Basic cell Bidirectional Seq2Seq-API Greedy
- LSTM Bidirectional Seq2Seq-API Greedy
- GRU Bidirectional Seq2Seq-API Greedy
- Basic cell Seq2Seq-manual + Luong Attention
- LSTM Seq2Seq-manual + Luong Attention
- GRU Seq2Seq-manual + Luong Attention
- Basic cell Seq2Seq-manual + Bahdanau Attention
- LSTM Seq2Seq-manual + Bahdanau Attention
- GRU Seq2Seq-manual + Bahdanau Attention
- LSTM Bidirectional Seq2Seq-manual + Luong Attention
- GRU Bidirectional Seq2Seq-manual + Luong Attention
- LSTM Bidirectional Seq2Seq-manual + Bahdanau Attention
- GRU Bidirectional Seq2Seq-manual + Bahdanau Attention
- LSTM Bidirectional Seq2Seq-manual + backward Bahdanau + forward Luong
- GRU Bidirectional Seq2Seq-manual + backward Bahdanau + forward Luong
- LSTM Seq2Seq-API Greedy + Luong Attention
- GRU Seq2Seq-API Greedy + Luong Attention
- LSTM Seq2Seq-API Greedy + Bahdanau Attention
- GRU Seq2Seq-API Greedy + Bahdanau Attention
- LSTM Seq2Seq-API Beam Decoder
- GRU Seq2Seq-API Beam Decoder
- LSTM Bidirectional Seq2Seq-API + Luong Attention + Beam Decoder
- GRU Bidirectional Seq2Seq-API + Luong Attention + Beam Decoder
- LSTM Bidirectional Seq2Seq-API + backward Bahdanau + forward Luong + Stack Bahdanau Luong Attention + Beam Decoder
- GRU Bidirectional Seq2Seq-API + backward Bahdanau + forward Luong + Stack Bahdanau Luong Attention + Beam Decoder
- Bytenet
- LSTM Seq2Seq + tf.estimator
- Capsule layers + LSTM Seq2Seq-API Greedy
- Capsule layers + LSTM Seq2Seq-API + Luong Attention + Beam Decoder
- LSTM Bidirectional Seq2Seq-API + backward Bahdanau + forward Luong + Stack Bahdanau Luong Attention + Beam Decoder + Dropout + L2
- DNC Seq2Seq
- LSTM Bidirectional Seq2Seq-API + Luong Monotic Attention + Beam Decoder
- LSTM Bidirectional Seq2Seq-API + Bahdanau Monotic Attention + Beam Decoder
- End-to-End Memory Network + Basic cell
- End-to-End Memory Network + LSTM cell
- Attention is all you need
- Transformer-XL
- Attention is all you need + Beam Search
- Transformer-XL + LSTM
- GPT-2 + LSTM
- Fairseq
- Conv-Encoder + LSTM
- Tacotron + Greedy decoder
- Tacotron + Beam decoder
- Google NMT
Neural Machine Translation (English to Vietnam)
- Seq2Seq-manual
- Seq2Seq-API Greedy
- Bidirectional Seq2Seq-manual
- Bidirectional Seq2Seq-API Greedy
- Bidirectional Seq2Seq-manual + backward Bahdanau + forward Luong
- Bidirectional Seq2Seq-API + backward Bahdanau + forward Luong + Stack Bahdanau Luong Attention + Beam Decoder
- Bytenet
- Capsule layers + LSTM Seq2Seq-API + Luong Attention + Beam Decoder
- End-to-End Memory Network
- Attention is All you need
Complete list (53 notebooks)
- Basic cell Seq2Seq-manual
- LSTM Seq2Seq-manual
- GRU Seq2Seq-manual
- Basic cell Seq2Seq-API Greedy
- LSTM Seq2Seq-API Greedy
- GRU Seq2Seq-API Greedy
- Basic cell Bidirectional Seq2Seq-manual
- LSTM Bidirectional Seq2Seq-manual
- GRU Bidirectional Seq2Seq-manual
- Basic cell Bidirectional Seq2Seq-API Greedy
- LSTM Bidirectional Seq2Seq-API Greedy
- GRU Bidirectional Seq2Seq-API Greedy
- Basic cell Seq2Seq-manual + Luong Attention
- LSTM Seq2Seq-manual + Luong Attention
- GRU Seq2Seq-manual + Luong Attention
- Basic cell Seq2Seq-manual + Bahdanau Attention
- LSTM Seq2Seq-manual + Bahdanau Attention
- GRU Seq2Seq-manual + Bahdanau Attention
- LSTM Bidirectional Seq2Seq-manual + Luong Attention
- GRU Bidirectional Seq2Seq-manual + Luong Attention
- LSTM Bidirectional Seq2Seq-manual + Bahdanau Attention
- GRU Bidirectional Seq2Seq-manual + Bahdanau Attention
- LSTM Bidirectional Seq2Seq-manual + backward Bahdanau + forward Luong
- GRU Bidirectional Seq2Seq-manual + backward Bahdanau + forward Luong
- LSTM Seq2Seq-API Greedy + Luong Attention
- GRU Seq2Seq-API Greedy + Luong Attention
- LSTM Seq2Seq-API Greedy + Bahdanau Attention
- GRU Seq2Seq-API Greedy + Bahdanau Attention
- LSTM Seq2Seq-API Beam Decoder
- GRU Seq2Seq-API Beam Decoder
- LSTM Bidirectional Seq2Seq-API + Luong Attention + Beam Decoder
- GRU Bidirectional Seq2Seq-API + Luong Attention + Beam Decoder
- LSTM Bidirectional Seq2Seq-API + backward Bahdanau + forward Luong + Stack Bahdanau Luong Attention + Beam Decoder
- GRU Bidirectional Seq2Seq-API + backward Bahdanau + forward Luong + Stack Bahdanau Luong Attention + Beam Decoder
- Bytenet
- LSTM Seq2Seq + tf.estimator
- Capsule layers + LSTM Seq2Seq-API Greedy
- Capsule layers + LSTM Seq2Seq-API + Luong Attention + Beam Decoder
- LSTM Bidirectional Seq2Seq-API + backward Bahdanau + forward Luong + Stack Bahdanau Luong Attention + Beam Decoder + Dropout + L2
- DNC Seq2Seq
- LSTM Bidirectional Seq2Seq-API + Luong Monotic Attention + Beam Decoder
- LSTM Bidirectional Seq2Seq-API + Bahdanau Monotic Attention + Beam Decoder
- End-to-End Memory Network + Basic cell
- End-to-End Memory Network + LSTM cell
- Attention is all you need
- Transformer-XL
- Attention is all you need + Beam Search
- Fairseq
- Conv-Encoder + LSTM
- Bytenet Greedy
- Residual GRU Bidirectional Seq2Seq-API Greedy
- Google NMT
- Dilated Seq2Seq
Embedded
- Word Vector using CBOW sample softmax
- Word Vector using CBOW noise contrastive estimation
- Word Vector using skipgram sample softmax
- Word Vector using skipgram noise contrastive estimation
- Lda2Vec Tensorflow
- Supervised Embedded
- Triplet-loss + LSTM
- LSTM Auto-Encoder
- Batch-All Triplet-loss LSTM
- Fast-text
- ELMO (biLM)
POS-Tagging
- Bidirectional RNN + Bahdanau Attention + CRF
- Bidirectional RNN + Luong Attention + CRF
- Bidirectional RNN + CRF
Entity-Tagging
- Bidirectional RNN + Bahdanau Attention + CRF
- Bidirectional RNN + Luong Attention + CRF
- Bidirectional RNN + CRF
- Char Ngrams + Bidirectional RNN + Bahdanau Attention + CRF
- Char Ngrams + Residual Network + Bahdanau Attention + CRF
- Char Ngrams + Attention is you all Need + CRF
Dependency-Parser
- Bidirectional RNN + Bahdanau Attention + CRF
- Bidirectional RNN + Luong Attention + CRF
- Residual Network + Bahdanau Attention + CRF
- Residual Network + Bahdanau Attention + Char Embedded + CRF
- Attention is all you need + CRF
Question-Answers
- End-to-End Memory Network + Basic cell
- End-to-End Memory Network + GRU cell
- End-to-End Memory Network + LSTM cell
- Dynamic Memory
Stemming
- LSTM + Seq2Seq + Beam
- GRU + Seq2Seq + Beam
- LSTM + BiRNN + Seq2Seq + Beam
- GRU + BiRNN + Seq2Seq + Beam
- DNC + Seq2Seq + Greedy
- BiRNN + Bahdanau + Copynet
Abstractive Summarization
- LSTM Seq2Seq using topic modelling
- LSTM Seq2Seq + Luong Attention using topic modelling
- LSTM Seq2Seq + Beam Decoder using topic modelling
- LSTM Bidirectional + Luong Attention + Beam Decoder using topic modelling
- LSTM Seq2Seq + Luong Attention + Pointer Generator
- Bytenet
- Pointer-Generator + Bahdanau, https://github.com/xueyouluo/my_seq2seq
- Copynet
- Pointer-Generator + Luong, https://github.com/xueyouluo/my_seq2seq
- Dilated Seq2Seq
- Dilated Seq2Seq + Self Attention
Extractive Summarization
- Skip-thought Vector
- Residual Network using Atrous CNN
- Residual Network using Atrous CNN + Bahdanau Attention
OCR (optical character recognition)
- CNN + LSTM RNN
Speech to Text
- Tacotron, https://github.com/Kyubyong/tacotron_asr
- Bidirectional RNN + Greedy CTC
- Bidirectional RNN + Beam CTC
- Seq2Seq + Bahdanau Attention + Beam CTC
- Seq2Seq + Luong Attention + Beam CTC
- Bidirectional RNN + Attention + Beam CTC
- Wavenet
- CNN encoder + RNN decoder + Bahdanau Attention
- CNN encoder + RNN decoder + Luong Attention
- Dilation CNN + GRU Bidirectional
- Deep speech 2
- Pyramid Dilated CNN
Text to Speech
- Tacotron, https://github.com/Kyubyong/tacotron
- Fairseq + Dilated CNN vocoder
- Seq2Seq + Bahdanau Attention
- Seq2Seq + Luong Attention
- Dilated CNN + Monothonic Attention + Dilated CNN vocoder
- Dilated CNN + Self Attention + Dilated CNN vocoder
- Deep CNN + Monothonic Attention + Dilated CNN vocoder
- Deep CNN + Self Attention + Dilated CNN vocoder
Old-to-Young Vocoder
- Dilated CNN
Generator
- Character-wise RNN + LSTM
- Character-wise RNN + Beam search
- Character-wise RNN + LSTM + Embedding
- Word-wise RNN + LSTM
- Word-wise RNN + LSTM + Embedding
- Character-wise + Seq2Seq + GRU
- Word-wise + Seq2Seq + GRU
- Character-wise RNN + LSTM + Bahdanau Attention
- Character-wise RNN + LSTM + Luong Attention
- Word-wise + Seq2Seq + GRU + Beam
- Character-wise + Seq2Seq + GRU + Bahdanau Attention
- Word-wise + Seq2Seq + GRU + Bahdanau Attention
- Character-wise Dilated CNN + Beam search
- Transformer + Beam search
- Transformer XL + Beam search
Topic Generator
- TAT-LSTM
- TAV-LSTM
- MTA-LSTM
Language-detection
- Fast-text Char N-Grams
Text Similarity
- Character wise similarity + LSTM + Bidirectional
- Word wise similarity + LSTM + Bidirectional
- Character wise similarity Triplet loss + LSTM
- Word wise similarity Triplet loss + LSTM
Text Augmentation
- Pretrained Glove
- GRU VAE-seq2seq-beam TF-probability
- LSTM VAE-seq2seq-beam TF-probability
- GRU VAE-seq2seq-beam + Bahdanau Attention TF-probability
- VAE + Deterministic Bahdanau Attention, https://github.com/HareeshBahuleyan/tf-var-attention
- VAE + VAE Bahdanau Attention, https://github.com/HareeshBahuleyan/tf-var-attention
Attention
- Bahdanau
- Luong
- Hierarchical
- Additive
- Soft
- Attention-over-Attention
- Bahdanau API
- Luong API
Miscellaneous
- Attention heatmap on Bahdanau Attention
- Attention heatmap on Luong Attention
Not-deep-learning
- Markov chatbot
- Decomposition summarization (3 notebooks)