ML Algorithms From Scratch: Bayesian Inference and Deep Learning

Chp02: Markov Chain Monte Carlo (MCMC)

• Estimate Pi: Monte Carlo estimate of Pi
• Binomial Tree Model: Monte Carlo simulation of binomial stock price
• Random Walk: self-avoiding random walk
• Gibbs Sampling: Gibbs sampling of multivariate Gaussian distribution
• Metropolis-Hastings Sampling: Metropolis-Hastings sampling of multivariate Gaussian mixture
• Importance Sampling: importance sampling for finding expected value of a function

Chp03: Variational Inference (VI)

• Mean Field VI: image denoising in Ising model

Chp04: Software Implementation

• Subset Generation: a complete search algorithm
• Fractional Knapsack: a greedy algorithm
• Binary Search: a divide and conquer algorithm
• Binomial Coefficients: a dynamic programming algorithm

Chp05: Classification Algorithms

• Perceptron: perceptron algorithm
• SVM: support vector machine
• SGD-LR: stochastic gradient descent logistic regression
• Naive Bayes: Bernoulli Naive Bayes algorithm
• CART: decision tree classification algorithm

Chp06: Regression Algorithms

• KNN: K-Nearest Neighbors regression
• BLR: Bayesian linear regression
• HBR: Hierarchical Bayesian regression
• GPR: Gaussian Process regression

Chp07: Selected Supervised Learning Algorithms

• Page Rank: Google page rank algorithm
• HMM: EM algorithm for Hidden Markov Models
• Imbalanced Learning: Tomek Links, SMOTE
• Active Learning: LR
• Bayesian optimization: BO
• Ensemble Learning: Bagging, Boosting, Stacking

Chp08: Unsupervised Learning Algorithms

• DP-Means: Dirichlet Process (DP) K-Means
• EM-GMM: EM algorithm for Gaussian Mixture Models
• PCA: Principal Component Analysis
• t-SNE: t-SNE manifold learning

Chp09: Selected Unsupervised Learning Algorithms

• LDA: Variational Inference for Latent Dirichlet Allocation
• KDE: Kernel Density Estimator
• TPO: Tangent Portfolio Optimization
• ICE: Inverse Covariance Estimation
• SA: Simulated Annealing
• GA: Genetic Algorithm