Training the controller and getting stuck in local minima about world-models HOT 5 CLOSED

Hi SynecFD,
Have you looked at the results obtained with the VAE? In particular, have you checked that the images generated by the VAE looks like game images?
If the VAE is working correctly and you kept the same hyperparameters than ours for the controller, the training of the controller with CMA should work fine, whatever your hardware is.

from world-models.

Hi leonardblier,
I also thought about that and checked the sample images and they do look very much like the game. To make sure I retrained the VAE a couple of times. The best I could get was a test loss of about 31, which seemed high at first, but I don't have much to compare these numbers to anything and I know an AutoEncoder can never be perfect anyway.
I left the hyperparameters exactly the same, just decreased the amount of workers. Maybe a new version of pytorch or pycma breaks it? I do get warnings that some functions are deprecated when starting the controller.

I could try to rerun everything with an older version and see if that fixes it.

from world-models.

Hi SynecFD,
I am quite sure the pipeline worked with the hyperparameters in the README : --n-samples 4 --pop-size 4, but it was a long time ago.
If you want to try with other hyperparameter, the last time I trained it (for additional experiments), I used --n-samples 16 --pop-size 64 and it worked perfectly well (on a recent pytorch version). It will be slower, but you can try it. I reached 800 in about 50 epochs.
If this works better, I will change the README.
The number of workers will have no effect on the training (except on the time per epoch).

from world-models.

You are completely right! With those hyperparameters it is working now.
Like you said it's slow, but after 8 hours of training I'm already at 500. I consider this issue solved now. Thank you very much.

from world-models.

Which hyperparameters are you using to get 500? --n-samples 4 --pop-size 4 or --n-samples 16 --pop-size 64? And how many epoch do you train the controller ?

from world-models.