When replaying: "Episode XYZ is not replayed successfully. Skipping" about maniskill HOT 9 CLOSED

Hi Harry,

Thanks for trying our OSC interface. I will answer you questions separately.

1. Robot arm seems to be "wobbly": Our OSC interface includes an operational space action and a null space action. The null space action will affect the movements of the arm joints. From the video, I guess the "wobbly" arm is due to unstable null space actions. In addition, ManiSkill will clip the action in the joint space, this might be another reason.
2. The grippers would unexpectedly close: The fingers are directly controlled by users. In the case of single-arm robot, fingers are controlled by os_action[4:6] or joint_action[-2:].
3. Handle of the door would slip out of the gripper's grip: Firstly, our frictions should be large enough to open the door since we have provided successful demonstrations. Secondly, according to the video, it seems your robot move back a little bit before closing the fingers. I guess this might be the reason why your robot did not grasp the handle.

Let me know if you have further questions.

from maniskill.

Hi,

Thanks for your speedy response. After explicitly setting the fingers speed in the approaching phase, the unexpected closing issue was resolved.

However, we still observe the slipping problem after holding the robot's position when gripping the handle. please refer to the linked video.
https://drive.google.com/file/d/1gBoE_CIzK2b0qGqkRz-_4oda7gHQvlYO/view?usp=sharing

from maniskill.

Hi Harry,

Can you let me know the id of the object you are using? This can be accessed by print(env.level_config["layout"]["articulations"]). It would be helpful if you can provide the codes you used to open this drawer. Thanks!

from maniskill.

Hi, the print statement returns the following dictionary:
[{'name': 'cabinet', 'fix_base': True, 'position': [0, 0, 0], 'rotation': [1, 0, 0, 0], 'surface_material': 'object_material', 'num_target_links': 3, 'partnet_mobility_id': 1033, 'scale': 0.5483899271738176}]

The testing code I am using is:

        hand_forward = np.zeros(osc_interface.osc_dim)
        hand_forward[4] = -0.1
        hand_forward[5] = -0.1
        if phase_counter >= 100:
            hand_forward[0] = -0.2
        else:
            phase_counter += 1
            hand_forward[0] = 0
        print("PHASE 3 GRASPING AND BACKING UP")
        keep_close = osc_interface.operational_space_and_null_space_to_joint_space(
                            qpos, hand_forward, action[:osc_interface.null_space_dim])
        action = keep_close

phase_counter acts like a counter to count how many steps in the env have we grasped the knob. after 100 steps (which makes the grip firm, we would start backing up.

Regarding the unstable physics, would it be possible to fix it? Since the wobbly arm would really sabotage the accuracy pf the motion....

from maniskill.

Hi Harry,

We just checked our demonstration data, and found that the cabinet 1033 can be opened by grasping the the handle. Please check this video. The demonstration data has been provided, please check here.

About the "wobbly" arm: We guess it is relevant to the way we convert the operational space action to joint space action. Can you provide the full script you used? We can take a deeper look into it.

from maniskill.

Hi, just sent our code to reproduce the error to your gmail

from maniskill.

Update: The unstable physics was due to sampling from the action space, causing some issues in the NullSpace conversion.

Regarding the slipping problem, i understand that the demo showed a successful trial, but does it require the robot to approach from a different angle?

Also, we wonder if it is possible for you to share an example controller code in OSC (e.g. how you integrate an IK solver or how you execute the planned trajectory)? I think the main bottleneck here is the control module. SPecifically, you put dim=1 when you want to move the EE in x direction in your example in the test function of osc.py. How do I know which element in the os_action corresponds to which direction of the EE? thanks.

from maniskill.

Hi Harry,

Unfortunately, we currently do not have the plan to provide a motion planning + control baseline. But you are welcome to use our LfD baselines.

About the meaning of OSC action: the 6-dim OSC action is a body twist at the end effector origin frame ordered as [v omega].

from maniskill.

The unstable physics is because the user used a randomly sampled action in null space.

from maniskill.