mmaric27 / basicdnn Goto Github PK

Generic L-layer 'straight in Python' fully connected Neural Network implementation using numpy.

• Input data is supposed to be stacked in a matrix of n_x by m, where n_x is a number of input features for an example and m is the number of training examples.

• Output data is supposed to be stacked in a 1 by m matrix, where m is the number of training examples.

• Output layer can be either Sigmoid or Softmax classifier.

• Implemented activation functions: Sigmoid, ReLU, Leaky ReLU, Tanh, Softmax.

• Implemented weights initialization methods: zeros, random, He, Xavier.

• Implemented regularization methods: L2, Dropout.

• Implemented optimization methods: Mini-Batch Gradient Descent, Momentum, Adam

• Usage example (4 layer model with 3 hidden layers with 20, 7, 5 units and relu activation function and He weights initialization, and output layer with one unit, sigmoid function and random initialization):

  • MODEL = ((20, 'relu', 'he'), (7, 'relu', 'he'), (5, 'relu', 'he'), (1, 'sigmoid', 'random'))
  • parameters = L_layer_model(trainX, trainY, MODEL)
  • predictTrain = predict(trainX, parameters, trainY)
  • predictDev = predict(devX, parameters, devY)
  • predictTest = predict(testX, parameters, testY)

• Batch normalization to be implemented.