Comments (4)
You could check this implementation https://github.com/zmonoid/WorldModels/blob/master/model.py for more information at line 62
from world-models.
I am a bit rusty with this library but, if I remember correctly, you are right.
This library uses an output such that each Gaussian mixture has a defined μ
and σ
for each element of the array z_{t+1}
. I don't know about other implementations, but the original World Models uses the one you propose.
I don't think this approach is wrong, it is just a more restrictive one. If you use it with the proposed environments (carRacing and ViZDoom: Take Cover) you won't be able to see the difference.
from world-models.
I am a bit rusty with this library but, if I remember correctly, you are right.
This library uses an output such that each Gaussian mixture has a defined
μ
andσ
for each element of the arrayz_{t+1}
. I don't know about other implementations, but the original World Models uses the one you propose.I don't think this approach is wrong, it is just a more restrictive one. If you use it with the proposed environments (carRacing and ViZDoom: Take Cover) you won't be able to see the difference.
Hi Alex,
Thank you so much for the very quick response!
I found issue when I was training the rnn, the gmm loss was always relatively high like (0.95), but after I changed the definition and loss function, respectively , I found the gmm loss was reduced to -0.005 which means the likelihood is near 1. Do you have any experience how the value loss should be? And could you give me some suggestion to sample the latent variable from the distribution according to "temperature"?
Best regards
from world-models.
I found issue when I was training the rnn, the gmm loss was always relatively high like (0.95), but after I changed the definition and loss function, respectively , I found the gmm loss was reduced to -0.005 which means the likelihood is near 1. Do you have any experience how the value loss should be?
I have the feeling I'm missing too much information to provide a faithful answer to your question (which environment you are using, size of the latent space, algorithm for the vision module, etc). I'll try to give you some general advice:
-
First of all, yes, that new value you are getting doesn't seem right. You might need to check your code for any errors. If you are using ViZDoom as an environment, check the World Models original repo. It uses the metric you propose and they have a logfile for their experiments with viZDoom: Take Cover. They get a final loss around 0.98 after 400 epochs
-
Using this library with the original configuration, and the loss function you propose, I got a GMM loss of 1.05 on the carRacing game and 0.95 on ViZDoom: Take cover. I might be wrong about this one (this was a long time ago, I don't remember the exact details) but take these values as reference. I ran other environments and configurations, but my loss never varied as significantly as yours.
-
Pay a lot of attention to your vision algorithm and its resulting values. Sometimes you get very strange results from your memory module but the bug isn't there. The first thing that comes to mind is KL divergence of the VAE. The higher it is, the lower the loss of the memory module (but it doesn't mean the module is better).
-
Talking about KL divergence, the original paper uses "free bits", which I think was not implemented in this library (Have a look at these papers: paper 1, paper 2). It helps with the reconstructions of both test environments by setting a threshold on the KL divergence. Is your latent space informative enough?
And could you give me some suggestion to sample the latent variable from the distribution according to "temperature"?
Well, this is a completely different question. From my experience, there are a couple of things you need to know:
-
Using
τ<1
never helped. The controller just learnt to trick the simulation more consistently. -
Training offline (in a dream) was very VERY tricky. Models trained with the same configuration and architecture could yield very different results. And this affected the temperature value too. From my experience, values
1.2 < τ > 1.1
worked best.
from world-models.
Related Issues (20)
- Shouldn't this be outside the for loop? HOT 3
- Data generation script: No module named 'utils' HOT 3
- Different transform in trainvae.py & trainmdrnn.py HOT 2
- sleep(0.1) leads to infty loops HOT 3
- inconsisent MDRNN / MDRNNCell behavoir HOT 3
- Error training MD-rnn HOT 3
- Splitting of Train and Validation / Test set HOT 1
- one question about gmm_loss function HOT 1
- Possible error when predicting next action (class RolloutGenerator) HOT 7
- a multi-process problem in the controller
- Training the controller and getting stuck in local minima HOT 5
- Multiprocessing very slow HOT 2
- issue about gmm_loss HOT 2
- Controller Input HOT 1
- the train_controller always break off when trainning about 15min
- MDRNN losses extremely low due to numerical instability?
- problem about training VAE
- MDRNN doesn't train properly on carracing?
- Worker dying issue with controller training
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from world-models.