Questions about architecture details. about latent-pose-reenactment HOT 2 CLOSED

고맙습니다, 강열씨!

1. This is to stay a little bit safer from vanishing gradients. Consider a pixel where the ground truth value is 1.0. Then, without such range shift, the network will be trained to emit a large positive output value so that tanh(output) ≈ 1.0. The possible danger is that output can happen to be so large that the gradient of tanh at output will be too close to zero, hampering the training. After moving the range, gradients at extreme ground truth values become adequate while the output values are still limited.

   I've been using this trick since my very first experiments. If you remove it, I'm sure everything will still work. After all, many people successfully train generative nets with plain tanh. But I just haven't checked.

   Even more, I think you can even use plain linear activation (no sigmoid, tanh or anything) instead of this trick, and everything will still work.

2. It worked in Zakharov et al. So we borrowed it as-is from there to save on experiments. Today we realize that your concern is indeed quite reasonable, and that BNs or INs are likely to improve the system.

from latent-pose-reenactment.

Thank you for your kind answer!

from latent-pose-reenactment.