danielkelshaw / radialbnn Goto Github PK

When running the mnist_radial_bnn.py example on GPU there were some issues with CUDA - this should be easily fixed by ensuring that all Tensors are on the correct device.

At the moment the ELBO loss is calculated by running multiple forward passes through the network for each evaluation. This could potentially be sped up by running a multiple forward pass for each loss evaluation but with higher-dimensional tensors in order to represent additional samples from the posterior.

Although the theory remains very much the same, it would be interesting to see if this accelerated the training process at all.

Provide the general framework for the repository and ensure everything is set up appropriately.

At the moment the calculation of the ELBO is constrained within a class decorator which must be used when defining the model. It would be good if the user had an alternative way to access this loss calculation.

At the moment the implementation of the RadialLayer does not take into account any bias terms - it would be good if the user had the opportunity to include this in the training of the layers.

Provide an implementation for the a Radial Linear layer. This should follow the implementation described within the paper.

Note: Use a Gaussian prior for now.

At the moment there is a pure code implementation of the the RadialLayer - there should be some form of example to demonstrate how this is used and to ensure that it works as expected.

It would be nice to provide an overview of the code / theory in the README.md so that readers get a sense of what the project is all about - this can simply highlight the key outcomes from the paper.

At the moment there is only an example for classification - MNIST. It would be beneficial to show how a regression case might be developed as well.

At the moment the models can only be used for classification-type problems as the loss for the regression case cannot be calculated properly yet. In order to accommodate for the regression case, the NLL of the samples taken should be taken.