A Bayesian approach to training neural networks can come with benefits such as natural weight regularization, straightforward approaches for weight pruning, and making prediction more uncertain in regions with sparse training data. Previous work showed that feedforward networks could be trained in the Bayesian framework via a backpropagation strategy (Bayes by Backprop) and demonstrated the aforementioned benefits experimentally. A recent paper extended the method to Convolutional Neural Networks but did not explore whether said advantages carry over to CNNs. We evaluated them here.
Bayes by Backprop is explained in the ICML 2015 paper Weight Uncertainty in Neural Networks. We modify the GitHub code provided by the authors of Bayesian Convolutional Neural Networks, which was available on GitHub but has been removed.
(Fall 2018)
Ryan L. Murphy, Jincheng Bai.
We use PyTorch version 0.4.1.