How are we using rewards in imitation learning? about pytorch-rl HOT 4 CLOSED

The reward is computed exactly as log of discriminator, as shown in this line:

from pytorch-rl.

The reward is computed exactly as log of discriminator, as shown in this line:

PyTorch-RL/gail/gail_gym.py

Line 99 in d94e147

return -math.log(discrim_net(state_action)[0].item())

This is exactly how the reward should be calculated, however everywhere it is written as the expectation of the derivative of the log term (See above images). Can you please tell me why the 'minus' sign is removed in the expectation term?

The D in this code is actually equivalent to the minus D in the original paper.

From the above two lines of code, it can be seen that the discriminator's update goal is to output 1 for the generated data g_o and 0 for the expert data e_o. The goal of the policy update should then be to minimize the output of the discriminator, i.e., to maximize the -log(D(g_o)) reward, which makes the so-called adversarial training.

from pytorch-rl.

Thank you for your response. For anyone else also wondering the same thing. Please check this line as well:

from pytorch-rl.

The reward is computed exactly as log of discriminator, as shown in this line:

PyTorch-RL/gail/gail_gym.py

Line 99 in d94e147

return -math.log(discrim_net(state_action)[0].item())

This is exactly how the reward should be calculated, however everywhere it is written as the expectation of the derivative of the log term (See above images). Can you please tell me why the 'minus' sign is removed in the expectation term?

from pytorch-rl.