This repo contains code to reproduce experiments in paper "Direct Feedback Alignment Provides Learning in Deep Neural Networks" (https://arxiv.org/abs/1609.01596)

This code and readme is copied and modified based on https://github.com/eladhoffer/ConvNet-torch (Deep Networks on classification tasks using Torch)

Supported datasets are {Cifar10/100, STL10, SVHN, MNIST}

##Data You can get the needed data using @soumith's repo: https://github.com/soumith/cifar.torch.git

##Dependencies

• Torch (http://torch.ch)
• "DataProvider.torch" (https://github.com/eladhoffer/DataProvider.torch) for DataProvider class.
• "cudnn.torch" (https://github.com/soumith/cudnn.torch) for faster training. Can be avoided by changing "cudnn" to "nn" in models.
• "dpnn" (https://github.com/Element-Research/dpnn) for maxnorm constraints on weights
• "unsup" (https://github.com/koraykv/unsup) for whitening of data

To install all dependencies (assuming torch is installed) use:

luarocks install https://raw.githubusercontent.com/eladhoffer/eladtools/master/eladtools-scm-1.rockspec
luarocks install https://raw.githubusercontent.com/eladhoffer/DataProvider.torch/master/dataprovider-scm-1.rockspec
luarocks install dpnn
luarocks install unsup

##Training You can reproduce the best results for direct feedback-alignment for each dataset with:

th Main.lua -dataset MNIST -network mlp.lua -LR 1e-4 -eps 0.08

or,

th Main.lua -dataset Cifar10 -network conv.lua -LR 5e-5 -whiten

or,

th Main.lua -dataset Cifar100 -network conv.lua -LR 2.5e-5 -whiten

##Additional flags