Maybe a bug in the evaluation of DDPG about tianshou HOT 6 CLOSED

From my previous experience, (in Q-learning based algorithms) if using an eps in the training phase but zero eps in the test phase, the performance would drop down; but with smaller eps but greater than zero, it works well.
Sorry for the late reply. I'm busy with my graduation these days (and Tsinghua U is in Beijing :(

from tianshou.

From my previous experience, (in Q-learning based algorithms) if using an eps in the training phase but zero eps in the test phase, the performance would drop down; but with smaller eps but greater than zero, it works well.
Sorry for the late reply. I'm busy with my graduation these days (and Tsinghua U is in Beijing :(

Oh, poor you.
Maybe it should leave an interface to users, so that they could decide whether to add an action noise in evaluation?
Hope you guys are fine.

from tianshou.

Yes, I agree with you. Maybe you can submit another pull request once you come up with the interface design.

from tianshou.

Sorry for the late replay. I've submitted a pull request with the interface design.

from tianshou.

Is it necessary to decrease noise as the training process goes on (in ddpg policy)? Can I do this in tianshou?
在ddpg中有必要将噪声衰减吗？在天授框架中可以这样做吗？

from tianshou.

Is it necessary to decrease noise as the training process goes on (in ddpg policy)? Can I do this in tianshou?

As you wish. Just modify the Gaussian Noise class and add a counter, each __call__ will increase the counter and decrease sigma a little.

from tianshou.