Manifold Regularized Deep Neural Networks written in Python using numpy and gnumpy to run on GPUs.

The base of this code (pre-trained deep neural networks) is taken from gdbn code written by George Dahl. http://www.cs.toronto.edu/~gdahl/. I have modified the DNN training by adding manifold regularization to his code. For a detailed overview of Manifold Regularized Neural Networks (MRDNN), please see my paper from Interspeech 2014 here: http://www.ece.mcgill.ca/~vtomar/Publications/Interspeech2014-MRDNN.pdf . Please cite the paper if you use this code. I have also uploaded a pdf for the system architecture: MRDNN_architecture.pdf in the repository.

There is no separate documentation as of yet. But the code should be easy enough to understand with the inline comments.

• trainInps is an N x D-dimensional array. N feature vectors each having D dimensions
• labels is a vector of dimensionality N containing a label for each feature vector
• trainTargs converts the labels vector into numberic classes and 1-hot target vectors for the DNN or MRDNN training

Manifold based relationships are build in the file mrdnn/manifold.py and mrdnn/manifold_noPen.py

• The network reads in pre-computed affinity graphs for the manifold data. Each graph W is a N x N sparse matrix represented by two matrices:

• Dist: This is a N x k matrix (k = number of nearest neighbors of each vector). Each j\th row of Dist contains distance of vector x_j to k of it's nearest neighbors.
• Ind: This is also a N x k matrix. Each row contains the indices of the nearest neighbors saved in Dist.
• If both the intrinsic and penalty graphs are used, Dist and Ind for these graphs are differenciated by further suffices: Disti and Indi for the intrinsic graph and Distp and Indp for the penalty graph.

gnumpy: Is available in the helpers folder, but originally taken from: http://www.cs.toronto.edu/~tijmen/gnumpy.html

npmat: http://www.cs.toronto.edu/~ilya/npmat.py) (a non-gpu cudamat simulation)

cudamat: http://code.google.com/p/cudamat/ (for running on GPUs), or

See LICENSE file