PhasedLSTM-Keras

Keras implementation of Phased LSTM [https://arxiv.org/abs/1610.09513], from NIPS 2016.

Works both with Theano and Tensorflow backend (although Theano recommended, as 3x faster).

Classification performance compared to standard Keras LSTM for MNIST dataset:

Phased LSTM::

Epoch 1/20 60000/60000 [==============================] - 324s - loss: 2.0418 - acc: 0.2513

Epoch 2/20 60000/60000 [==============================] - 319s - loss: 1.7242 - acc: 0.3654

Epoch 3/20 60000/60000 [==============================] - 314s - loss: 1.7099 - acc: 0.3501

Epoch 4/20 60000/60000 [==============================] - 314s - loss: 1.5299 - acc: 0.4254

Epoch 5/20 60000/60000 [==============================] - 313s - loss: 1.2343 - acc: 0.5388

Epoch 6/20 60000/60000 [==============================] - 314s - loss: 1.1064 - acc: 0.5926

Epoch 7/20 60000/60000 [==============================] - 314s - loss: 1.0078 - acc: 0.6425

Epoch 8/20 60000/60000 [==============================] - 314s - loss: 0.9120 - acc: 0.6825

Epoch 9/20 60000/60000 [==============================] - 314s - loss: 0.8294 - acc: 0.7134

Epoch 10/20 60000/60000 [==============================] - 311s - loss: 0.7552 - acc: 0.7434

Epoch 11/20 60000/60000 [==============================] - 311s - loss: 0.6813 - acc: 0.7685

Epoch 12/20 60000/60000 [==============================] - 311s - loss: 0.6143 - acc: 0.7901

Epoch 13/20 60000/60000 [==============================] - 311s - loss: 0.5686 - acc: 0.8028

Epoch 14/20 60000/60000 [==============================] - 311s - loss: 0.5320 - acc: 0.8156

Epoch 15/20 60000/60000 [==============================] - 311s - loss: 0.5097 - acc: 0.8223

Epoch 16/20 60000/60000 [==============================] - 311s - loss: 0.4750 - acc: 0.8353

Epoch 17/20 60000/60000 [==============================] - 311s - loss: 0.4507 - acc: 0.8467

Epoch 18/20 60000/60000 [==============================] - 312s - loss: 0.4354 - acc: 0.8538

Epoch 19/20 60000/60000 [==============================] - 316s - loss: 0.4106 - acc: 0.8618

Epoch 20/20 60000/60000 [==============================] - 316s - loss: 0.3934 - acc: 0.8695

LSTM::

Epoch 1/20 60000/60000 [==============================] - 157s - loss: 2.2945 - acc: 0.1216

Epoch 2/20 60000/60000 [==============================] - 157s - loss: 2.0987 - acc: 0.2275

Epoch 3/20 60000/60000 [==============================] - 157s - loss: 1.9601 - acc: 0.2926

Epoch 4/20 60000/60000 [==============================] - 157s - loss: 1.8418 - acc: 0.3247

Epoch 5/20 60000/60000 [==============================] - 157s - loss: 2.0860 - acc: 0.2619

Epoch 6/20 60000/60000 [==============================] - 157s - loss: 2.1297 - acc: 0.2225

Epoch 7/20 60000/60000 [==============================] - 157s - loss: 1.8556 - acc: 0.3287

Epoch 8/20 60000/60000 [==============================] - 157s - loss: 1.8428 - acc: 0.3344

Epoch 9/20 60000/60000 [==============================] - 158s - loss: 1.8119 - acc: 0.3219

Epoch 10/20 60000/60000 [==============================] - 158s - loss: 1.8159 - acc: 0.3246

Epoch 11/20 60000/60000 [==============================] - 158s - loss: 1.9290 - acc: 0.2554

Epoch 12/20 60000/60000 [==============================] - 158s - loss: 1.7843 - acc: 0.3047

Epoch 13/20 60000/60000 [==============================] - 158s - loss: 1.7623 - acc: 0.3371

Epoch 14/20 60000/60000 [==============================] - 158s - loss: 1.6016 - acc: 0.4079

Epoch 15/20 60000/60000 [==============================] - 158s - loss: 1.5954 - acc: 0.3985

Epoch 16/20 60000/60000 [==============================] - 157s - loss: 1.6393 - acc: 0.3823

Epoch 17/20 60000/60000 [==============================] - 157s - loss: 1.6186 - acc: 0.3939

Epoch 18/20 60000/60000 [==============================] - 157s - loss: 1.6276 - acc: 0.3835

Epoch 19/20 60000/60000 [==============================] - 157s - loss: 1.6557 - acc: 0.3684

Epoch 20/20 60000/60000 [==============================] - 157s - loss: 1.8699 - acc: 0.3258