Machine learning algorithm fundamental knowledge, Including code implementation notes and experiments. Most of the works are based on the assignment in Columbia University, See Acknowledgement of each notebook

• Implementing feedforward neural network with numpy
• Implementing Backpropogation to train the model
• Hand-write gradient calculation for backpropogation

• Implementation of common optimizers:
  • Gradient Descent, SGD, SGD + Momentum, AdaGrad, ADAM
  • Intuition behind each optimizer
• Test the performance of each optimizers on test functions:
  • bowl, mult, monkey, matyas, Easom
• To test and compare each optimizer when
  • Contains Local mins
  • Contains Saddle
  • Hyperparameter is wrong

• Fundamental knowledge of convolution
• Implement Conv2D with Numpy
• Applications and classic filters including
  • Image Denoising
  • Edge Detection
• Application of Fourier transform on image
  • Convolution in time space is multiplication in frequency space
  • Convolution in frequency space is multiplication in time space
  • When Kernel is large, multiplication in Fourier space can enhance efficiency

• Implement and test Matching Network and Prototypical Network
• N-way, K-shot classification on the Omniglot dataset
• Personally, I prefer to call it Convolutional KNN, because under the hood the model finish the task by:
  • Train a Conv encoder until the supporting sets are seperable in embedding space
  • Then use the encoder to map the query image into embedding space and run KNN with supporting set in the embedding space to generate the prediction
  • The only difference for Matching Network and Prototypical Network are the metric used for KNN

• Common Generative adversarial network on Minist dataset, nothing special

• Common Variational auto encoder on Minist dataset, nothing special