A3C loss functions about deeprl-agents HOT 1 CLOSED

Hi Gabriel, this is actually a feature, in a strange way. In A3C the value loss is often scaled by some coefficient in order to ensure smaller (and hopefully more stable) changes to the value function. The first 0.5 is part of the value loss itself, while the 0.5 in the full loss function is the regularizer parameter, similar to the 0.01 the entropy is multiplied by. I have actually seen the value loss scaled down to as much as 0.1 in some cases. In practice the separation can seem somewhat arbitrary though.

from deeprl-agents.