ermongroup / a-nice-mc Goto Github PK

Hi apologies @daniellevy, @tachim, @kuleshov, @llonchj ,
I am getting errors on running some of the examples.

(myenv) [andrewcz@andrewcz-pc examples]$ python nice_synthetic.py
Traceback (most recent call last):
File "nice_synthetic.py", line 13, in
from a_nice_mc.objectives.bayes_logistic_regression.synthetic import Synthetic
ModuleNotFoundError: No module named 'a_nice_mc.objectives'

Im not sure what im doing wrong.
Many thanks,
Best,
Andrew

Hi,

I want to try to use a-nice-mc to sample from the posterior weights of a neural net but in my initial experiments for a relatively small net I find that I'm not able to learn to draw reasonable samples even after 60000 training iterations.

I'm training a one-layer feedforward net with a hidden width of 50 units to perform regression on the Boston housing data-set.

I think the issue is that bootstrapping from the randomly initialised model is inadequate for such a high dimensional space. I'm going to try to bootstrap instead from samples drawn using HMC.

Do you have any advice or thoughts on this before I embark? Would you like to include the bootstrap in the repo?

thanks,

Raza

First of all, very nice work! @jiamings I'm not sure where to ask this question so I'm posting it here. My main concern is that feed forward neural networks are bad at extrapolation. Could fitting the proposal distribution to certain samples, e.g. the bootstrapped samples described in the paper, hampers ergodicity?

https://github.com/ermongroup/a-nice-mc/blob/master/a_nice_mc/utils/hmc.py#L108
For different step, it seems that you use the same stepsize. I guess stepsize should be replaced with s.
@jiamings