pku-epic / unidexgrasp2 Goto Github PK

Hi, thanks for the great work. Could I know when the code will be released?

# Record the transition
self.storage.add_transitions(current_obs, actions_expert, rews, dones)
current_obs.copy_(next_obs)

# value_net
if self.apply_value_net:
    self.ppo_buffer.add_transitions(
        current_obs, current_states, actions, rews, 
        dones, values, actions_log_prob, mu, sigma,
    )
    current_states.copy_(next_states)

the current_obs variable updates before adding transitions to ppo_buffer, so the ppo_buffer actually adds the next_obs instead the current_obs.

Hi, I have a question about dagger-value algorithm:
when updating value network, why do you use torch.max() to get the larger loss?

What's the meaning comparing these two losses? In my understanding, using clipped value loss is to keep the training procedure stable, but in that case why is it max not min, or, why not just use value_losses_clipped directly?

In above code, the target_values_batch is the student critic value before learning epoches, value_batch is student critic value during learning epoches.