Clarification about robot_obs in custom Gym environments about assistive-gym HOT 2 CLOSED

Hi there,

Good question! The observation does include the robot's joint angles (for controllable actuators), which is stored in the variable robot_right_joint_positions in this case above. As for the other elements in the observation, they help the robot infer the state of the world a bit easier. Technically all the robot needs to know about is its joint angles and a target goal, but then that requires the control model (in this case a neural network) to also learn how to do forward kinematics to compute how far the robot's end effector is from some goal. Instead, we just directly compute the forward kinematics to get the robot gripper position/orientation and provide that to the model to make learning a bit easier. For a quick validation of this, you could train simple controllers with and without this information in the observation and check how long it takes for them to learn the task.

self.target_pos is important to give the robot as an observation since that its the target goal for the robot to reach towards. If that information is not given to the robot, then the robot has no idea where it should move towards. Of course the reward will also be based on this target goal, but the robot only gets 'rewards' during training in simulation, not during test time. In practice, it is quite common for robots to know where their target goal is (otherwise they would just be moving around randomly!). For example, if helping someone eat food, the robot should know where the person's mouth is at each time step, which can be provided by off-the-shelf face detection software.

Hope this helps!

from assistive-gym.

Thanks for the clarifications! I actually realised that Mujoco's reacher environment also uses the same forward kinematic trick.
And yes obviously, the robot must know the target position to be able to learn...
Thanks!

from assistive-gym.