Machine learning algorithm fundamental knowledge, Including code implementation notes and experiments. Most of the works are based on the assignment in Columbia University, See Acknowledgement of each notebook
- Implementing feedforward neural network with numpy
- Implementing Backpropogation to train the model
- Hand-write gradient calculation for backpropogation
- Implementation of common optimizers:
- Gradient Descent, SGD, SGD + Momentum, AdaGrad, ADAM
- Intuition behind each optimizer
- Test the performance of each optimizers on test functions:
- bowl, mult, monkey, matyas, Easom
- To test and compare each optimizer when
- Contains Local mins
- Contains Saddle
- Hyperparameter is wrong
- Fundamental knowledge of convolution
- Implement Conv2D with Numpy
- Applications and classic filters including
- Image Denoising
- Edge Detection
- Application of Fourier transform on image
- Convolution in time space is multiplication in frequency space
- Convolution in frequency space is multiplication in time space
- When Kernel is large, multiplication in Fourier space can enhance efficiency
- Implement and test Matching Network and Prototypical Network
- N-way, K-shot classification on the Omniglot dataset
- Personally, I prefer to call it Convolutional KNN, because under the hood the model finish the task by:
- Train a Conv encoder until the supporting sets are seperable in embedding space
- Then use the encoder to map the query image into embedding space and run KNN with supporting set in the embedding space to generate the prediction
- The only difference for Matching Network and Prototypical Network are the metric used for KNN
- Common Generative adversarial network on Minist dataset, nothing special
- Common Variational auto encoder on Minist dataset, nothing special