yaringal / bayesianrnn Goto Github PK

Hi, I am attempting to run the Examples/sentiment_lstm_regression.py however I am running into some issues: it keeps failing on the initialization of the predict_stochastic function in the first callback of modeltest.

First the function fails on X = models.standardize_X(X), due to the below error. Is this function supposed to be standardize(X)? I can't find a standardize_X in the latest Keras pull (I am using 1.0.3)?
AttributeError: 'module' object has no attribute 'standardize_X'

The function then also fails on defining the K function because neither model.X_test nor model.y_train are defined. The example doesn't have a test set passed into the fit model, so I'm not sure which variable this would be referencing.
K.function([self.model.X_test], [self.model.y_train])
AttributeError: 'Sequential' object has no attribute 'X_test'

If I remove the Modeltest call backs, the training works fine.

Any help you could provide would be greatly appreciated. Thanks!

I notice in your readme it states the Variational Dropout algorithm has been implemented in Keras’ RNN library.

I want to verify that it is implemented exactly as described in your paper (that the exact dropped out connections remain constant throughout training).

I’m implementing the encoder-decoder framework from the paper on Uber’s timeseries anomaly prediction model.

Thank you.

Hi Yarin,

Could I ask which version of TF/keras you used to run this? I am having problems running it with tensorflow=0.11.0 and keras 1.1.2. I see the following error

Exception: Error when checking model target: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 1 arrays but instead got the following list of 10496 arrays: [-0.64800038508682045, -0.64800038508682045, 1.1463771479409459, 1.1463771479409459, 1.1463771479409459, -0.48487515481156884, -0.48487515481156884, 0.058875612772602504, 0.058875612772602504, -0.4848...

Cheers,
Jack

Hi, thank you for your excellent work.
I found some weird parts in source code.
In main_new_dropout_SOTA_v3.lua, line 216
local x = state.data[state.pos + i - 1]
Isn't it should be
local x = state.data[state.pos + i - 1][b]
??
and line 218
if state.data[state.pos + j - 1] == x then
Isn't it should be
if state.data[state.pos + j - 1][b] == x then
??
Other LM demos have the same problem.
If batch words are compared, there is few chance that they are completely equal, therefore word dropouts are not tied. I am wondering whether your result in your paper use the same source code?