inconsisent MDRNN / MDRNNCell behavoir about world-models HOT 3 CLOSED

You are right, I'm fixing this asap. Thanks a lot ! If I'm not mistaken, this should not affect performance (the MDRNNCell is only used when training the controller, and in this case, we only rely on the hidden state of the RNN, not its outputs).

from world-models.

Cool, thanks for confirming!

Right. So in the original paper / implementation there was a variant where the MDRNN was used to predict the next state, in the "virtual" or "dream world" to train the controller. (So in that scenario it matters.)

BTW -- do you have any plans to implement the virtual environment in this repo? That would be really useful!

from world-models.

Yep indeed, this would impact training in the simulated environment, which is currently not implemented.
Both Léonard and I are quite busy at the moment, so we are not planning on implementing training in the dreamt environment in the near future. There is an untested facility to simulate rollouts using the trained VAE and MDN-RNN in envs/simulated_carracing.py. If you are willing to adapt this to allow for training in the dream, I would gladly review the PR !

from world-models.