Locomotion performence is poor about alf HOT 5 CLOSED

Hi, let me rerun the task with several random seeds and see how it goes.

from alf.

Thank you. Mention one thing. The only changed para is num_parallel_environments. I set it to 20.

from alf.

Thank you. Mention one thing. The only changed para is num_parallel_environments. I set it to 20.

I see. Yeah setting it to >1 envs will definitely decrease its sample efficiency. This means that now every rollout step generates 20 time steps while the SAC paper (with only 1 env) generates 1 time step every rollout step. So the ratio of training samples to rollout samples is now only 1/20. I suggest setting it back to 1 for reproducing purpose.

In your case, if num of envs=20, then you should probably train it longer (targeting at least 20M steps) to see its final performance.

from alf.

I get it. Before, I thought num_parallel_environments just influence the traing efficiency, but would not affect the training performence. Thank you very much.

from alf.

I get it. Before, I thought num_parallel_environments just influence the traing efficiency, but would not affect the training performence. Thank you very much.

Update:

Out of four random seeds, three are able to get 6000 in 3M steps. One unlucky seed only gets about 3000. (BTW: this run uses grid_search.py for 4 repeats).

from alf.