paraschopra / bayesian-neural-network-mnist Goto Github PK

Hi! I really like your work and I'm trying to do something similar.
I didn't catch how do you 'predict' uncertainty. For sure I'm missing something but for me it's not clear the relation between the 50 percentile, the threshold value (0.2) and the "refusing to predict". Thanks

Hi,
In model(x_data, y_data), lhat = log_softmax(lifted_reg_model(x_data)).

In give_uncertainities(x), yhats = [F.log_softmax(model(x.view(-1,28*28)).data, 1).detach().numpy() for model in sampled_models]

Does it mean you use log_softmax twice?

Using updated versions of torch, torchvision, and pyro dependencies, an error (below) occurs in the instance of SVI where the event_dims between the model and guide disagree at site 'module$$$out.weight': 0 vs 1. Additionally, .independent() is deprecated and it is recommended that it be changed its replacement, .to_event(), in the following line of the guide: " outw_prior = Normal(loc=outw_mu_param, scale=outw_sigma_param).independent(1)". Unfortunately, changing from .independent(1) to .to_event(1) in the guide does not rectify the event_dims mismatch error.

Please help. I would very much like to be able to use your Bayesian Neural Network script as it deals with rejecting untrained classes in the test data according to probability.

P.S. On a side note regarding a previously discussed issue, the MNIST data is inaccessible using torchvision's datasets.MNIST as used in bnn.ipynb. I checked other image datasets such as Fashion-MNIST, and that was readily available. Currently, the MNIST data can be obtained with this command: "!wget www.di.ens.fr/~lelarge/MNIST.tar.gz".

Error Messages:
/usr/local/lib/python3.7/dist-packages/pyro/primitives.py:451: FutureWarning: The random_module primitive is deprecated, and will be removed in a future release. Use pyro.nn.Module to create Bayesian modules from torch.nn.Module instances.
"modules from torch.nn.Module instances.", FutureWarning)

ValueError Traceback (most recent call last)
in ()
6 for batch_id, data in enumerate(train_loader):
7 # calculate the loss and take a gradient step
----> 8 loss += svi.step(data[0].view(-1,28*28), data[1])
9 normalizer_train = len(train_loader.dataset)
10 total_epoch_loss_train = loss / normalizer_train

/usr/local/lib/python3.7/dist-packages/pyro/infer/svi.py in step(self, *args, **kwargs)
126 # get loss and compute gradients
127 with poutine.trace(param_only=True) as param_capture:
--> 128 loss = self.loss_and_grads(self.model, self.guide, *args, **kwargs)
129
130 params = set(site["value"].unconstrained()

/usr/local/lib/python3.7/dist-packages/pyro/infer/trace_elbo.py in loss_and_grads(self, model, guide, *args, **kwargs)
129 loss = 0.0
130 # grab a trace from the generator
--> 131 for model_trace, guide_trace in self._get_traces(model, guide, args, kwargs):
132 loss_particle, surrogate_loss_particle = self._differentiable_loss_particle(model_trace, guide_trace)
133 loss += loss_particle / self.num_particles

/usr/local/lib/python3.7/dist-packages/pyro/infer/elbo.py in _get_traces(self, model, guide, args, kwargs)
168 else:
169 for i in range(self.num_particles):
--> 170 yield self._get_trace(model, guide, args, kwargs)

/usr/local/lib/python3.7/dist-packages/pyro/infer/trace_elbo.py in _get_trace(self, model, guide, args, kwargs)
56 """
57 model_trace, guide_trace = get_importance_trace(
---> 58 "flat", self.max_plate_nesting, model, guide, args, kwargs)
59 if is_validation_enabled():
60 check_if_enumerated(guide_trace)

/usr/local/lib/python3.7/dist-packages/pyro/infer/enum.py in get_importance_trace(graph_type, max_plate_nesting, model, guide, args, kwargs, detach)
48 graph_type=graph_type).get_trace(*args, **kwargs)
49 if is_validation_enabled():
---> 50 check_model_guide_match(model_trace, guide_trace, max_plate_nesting)
51
52 guide_trace = prune_subsample_sites(guide_trace)

/usr/local/lib/python3.7/dist-packages/pyro/util.py in check_model_guide_match(model_trace, guide_trace, max_plate_nesting)
252 if model_site["fn"].event_dim != guide_site["fn"].event_dim:
253 raise ValueError("Model and guide event_dims disagree at site '{}': {} vs {}".format(
--> 254 name, model_site["fn"].event_dim, guide_site["fn"].event_dim))
255
256 if hasattr(model_site["fn"], "shape") and hasattr(guide_site["fn"], "shape"):

ValueError: Model and guide event_dims disagree at site 'module$$$out.weight': 0 vs 1

When working with random data,

test_batch(images_random, labels_random)

multiple runs interrupted with this small snafu:

Summary
Total images:  100
Predicted for:  0

---------------------------------------------------------------------------
ZeroDivisionError                         Traceback (most recent call last)
<timed eval> in <module>

<ipython-input-17-df814c720b20> in test_batch(images, labels, plot)
     81         print("Total images: ",len(labels))
     82         print("Predicted for: ",predicted_for_images)
---> 83         print("Accuracy when predicted: ",correct_predictions/predicted_for_images)
     84 
     85     return len(labels), correct_predictions, predicted_for_images

ZeroDivisionError: division by zero

Consider add a try-except block to catch the ZeroDivisionError and handle this?

Of course, this still drives your overall point correctly - but for someone simply skimming the code this might look like a bug.

Or alternatively, add the following line to your notebook please?

!rm -rvf ./not-mnist/.DS_Store