RETURNN paper 2016, RETURNN paper 2018.

RETURNN - RWTH extensible training framework for universal recurrent neural networks, is a Theano/TensorFlow-based implementation of modern recurrent neural network architectures. It is optimized for fast and reliable training of recurrent neural networks in a multi-GPU environment.

Features include:

• Mini-batch training of feed-forward neural networks
• Sequence-chunking based batch training for recurrent neural networks
• Long short-term memory recurrent neural networks including our own fast CUDA kernel
• Multidimensional LSTM (GPU only, there is no CPU version)
• Memory management for large data sets
• Work distribution across multiple devices
• Flexible and fast architecture which allows all kinds of encoder-attention-decoder models

Please read the documentation for more information.

There are some example demos in /demos which work on artifically generated data, i.e. they should work as-is.

There are some real-world examples here.

Some benchmark setups against other frameworks can be found here. The results are in the RETURNN paper 2016. Performance benchmarks of our LSTM kernel vs CuDNN and other TensorFlow kernels are here.