Git Product home page Git Product logo

bayesian-deep-learning-notes's Introduction

bayesian-deep-learning-notes

One-phrase-summary for Bayesian deep learning papers. We here organize these papers in the following categories. But some of them might have overlap.

(1). Uncertainty in deep learning

Model uncertainty in deep learning via Bayesian modelling by variatial inference etc.

  • [1705]. Concrete Dropout - [arxiv] [Note]
  • [1703]. Dropout Inference in Bayesian Neural Networks with Alpha-divergences - [arxiv] [Note]
  • [1703]. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? - [arxiv] [Note]
  • [2016]. Uncertainty in Deep Learning - [PDF] [Blog] [Note]
  • [1505]. Weight Uncertainty in Neural Networks - [arxiv] [Note]
  • [2015]. On Modern Deep Learning and Variational Inference - [NIPS] [Note]
  • [1995]. Bayesian learning for neural networks

(2). Probabilistic deep models

Use probabilistic model to imitate deep neural networks.

  • [1711]. Deep Gaussian Mixture Models - [arxiv]
  • [1411]. Deep Exponential Families - [arxiv] [Note]

(3). Probabilistic neural networks

Use probabilistic methods to do the inference in neural networks.

  • [1611]. Natural-Parameter Networks: A Class of Probabilistic Neural Networks - [arxiv] [Note]

(4). Approximate inference

Approximate inference or variational inference mostly is the building block for Bayesian deep learning. Variational inference: the main idea behind variational inference is to pick a family of distributions over the latent variables with its own parameters which is called variational parameters.

(4.1) General

  • [1712]. Vprop: Variational Inference using RMSprop - [arxiv] [Note]
  • [1709]. Perturbative Black Box Variational Inference - [arxiv] [Note]
  • [1703]. Conjugate-Computation Variational Inference : Converting Variational Inference in Non-Conjugate Models to Inferences in Conjugate Models - [arxiv] [Note]
  • [1611]. Variational Inference via ฯ‡-Upper Bound Minimization - [arxiv]
  • [1601]. Variational Inference: A Review for Statisticians - [arxiv]
  • [1401]. Black Box Variational Inference - [arxiv] [Note]
  • [2014]. Smoothed Gradients for Stochastic Variational Inference - [NIPS] [Note]
  • [1206]. Stochastic Variational Inference - [arxiv] [Note]
  • [2011]. Practical Variational Inference for Neural Networks - [NIPS]
  • [1999]. An Introduction to Variational Methods for Graphical Models - [PDF]

(4.2) Reparametrization trick in variational inference

  • [1506]. Variational Dropout and the Local Reparameterization Trick - [arxiv]
  • [1401]. Stochastic Backpropagation and Approximate Inference in Deep Generative Models - [arxiv]
  • [1312]. Auto-Encoding Variational Bayes - [arxiv] [Note]

(4.3) Others

(5) Continuous relaxation

Use continuous distribution to approximate discrete random variables, e.g. concrete distribution is a continuous distribution used to approximate discrete random variables.

  • [1611]. The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables - [arxiv]
  • [1611]. Categorical Reparameterization with Gumbel-Softmax - [arxiv]

(6) Bayesian neural network pruning

Sparse prior can be used to induce sparse weight or neuron in neural networks thus favor smaller network structure for mobile devices etc.

  • [1711]. Interpreting Convolutional Neural Networks Through Compression - [arXiv] [Note]
  • [1705]. Structural compression of convolutional neural networks based on greedy filter pruning - [arXiv] [Note]
  • [1705]. Structured Bayesian Pruning via Log-Normal Multiplicative Noise - [arxiv]
  • [1705]. Bayesian Compression for Deep Learning - [arxiv] [Note]
  • [1701]. Variational Dropout Sparsifies Deep Neural Networks - [arxiv]

Contribution

Any contribution is welcome. But notice that we need 'one phrase summary' to give an overview guidance to the readers RATHER THAN a list of papers. And please add yourself into the contributor list!

Contributors

bayesian-deep-learning-notes's People

Contributors

junlulocky avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.