farama-foundation / gymnasium-robotics Goto Github PK

View Code? Open in Web Editor NEW

439.0 12.0 75.0 381.37 MB

A collection of robotics simulation environments for reinforcement learning

Home Page: https://robotics.farama.org/

License: MIT License

Shell 0.07% Python 99.79% Dockerfile 0.15%

d4rl gymnasium mujoco reinforcement-learning robotics simulation

gymnasium-robotics's People

Contributors

Stargazers

Watchers

gymnasium-robotics's Issues

[Question] Adapting Fetch Robot for the FrankaKitchen Task

Adapting Fetch Robot for the FrankaKitchen Task

Hi,
I'm trying to add the Fetch Robot from the Fetch Environment into the Franka-Kitchen Task. I noticed that the dynamics (damping, friction) of the Fetch Robot are different from the ones in the urdf supplied by: https://github.com/ZebraDevs/fetch_ros/blob/melodic-devel/fetch_description/robots/fetch.urdf and also different from the Franka Robot. How can I make it as realistic as possible and also make it functional with the inverse kinematics library (similar to the Franka)?

Appreciate any suggestions.

[Question] Compatibility with stable-baselines3

Question

Hi,
how do I initialize a gymnasium-robotics environment such that it is compatible with stable-baselines3. E.g., I tried:

def make_env():
    env = gymnasium.make('FetchSlide-v2')
    return env
env = stable_baselines3.common.vec_env.DummyVecEnv([make_env])

which returns an error:

  File "/path/to/stable-baselines3/stable_baselines3/common/vec_env/util.py", line 68, in obs_space_info
    assert not hasattr(obs_space, "spaces"), f"Unsupported structured space '{type(obs_space)}'"
AssertionError: Unsupported structured space '<class 'gymnasium.spaces.dict.Dict'>'

I found this repo from Rodrigo de Lazcano: validate-mujoco-bindings-gym-robotics. However, it seems to use an older version of gymnasium-robotics (gym-robotics).
I couldn't find any more recent examples or documentation.

[Question] How to get the absolute position and velocities in the global coordinate system

I am working on multi-agent reacher with 2 links. How can I get the absolute position and velocities in the global coordinate system? Thank you for the help.

[Proposal] MinariDataset Generation

Proposal

Up-to-date, re-runnable expert Minari datasets for offline learning.

Motivation

Minari is now at a stage where datasets can be made and uploaded for experimental use by others.

Pitch

Have an expert policy perform 100k rollouts on each environment, save this buffer as a MinariDataset and upload it for public use.

Alternatives

If anyone thinks Minari needs to be improved before doing this, let me know.

Additional context

See https://github.com/Farama-Foundation/Minari/blob/main/tutorials/DatasetCreation/dataset_creation.py for an example of creating and uploading a MinariDataset.

Checklist

I have checked that there is no similar issue in the repo (required)

[Bug Report] mujoco_utils.get_joint_qvel() returning shifted values.

Hi, I think I found a small bug in the mujoco_utils.get_joint_qvel() function.

In line 202:
joint_addr = model.jnt_qposadr[joint_id]

the joint address is taken from jnt_qposadr but I think it should be jnt_dofadr like in mujoco_utils.set_joint_qvel().

This becomes a problem when qvel array has different size than qpos (e.g. when using constraints) as the function returns shifted values.

[Bug Report] `step` function in `franka_env.py` takes incorrect action

Hi,
Describe the bug
The step function in franka_env.py takes incorrect action in the non-IK controller (

Gymnasium-Robotics/gymnasium_robotics/envs/franka_kitchen/franka_env.py

Line 144 in c9a2e3d

self.data.ctrl[:] = self.actuation_center + action * self.actuation_range

). First, it denormalizes the input action from [-1, 1] range to each actuators' control range, and then calls do_simulation(action) with the normalized actions which is incorrect. It should input the denormalized action into do_simulation()

Code example
self.data.ctrl[:] = self.actuation_center + action * self.actuation_range
self.do_simulation(action, self.frame_skip)
It should be:
action = self.actuation_center + action * self.actuation_range
self.do_simulation(action, self.frame_skip)

[Question] How can I add another point in point.xml file?

Hello,

I'm doing multiagent research and wonder how to add another ball into the maze envs.

It seems that I need to change the point.xml file but I'm having problems rendering it after the change to this file.

Thank you so much!

[Question] Lib version mismatch?

Question

Hi!

I'm working with FrankaKitchen-v1 env and notice that the action which is not denormalized is passed to the function do_simulation in franka_env.py in my virtual env.

# Denormalize the input action from [-1, 1] range to the each actuators control range
self.data.ctrl[:] = self.actuation_center + action * self.actuation_range
self.do_simulation(action, self.frame_skip)
if self.render_mode == "human":
    self.render()

Meanwhile, in this line, the denormalized action is correctly passed to the function do_simulation.

I have installed the main branch of Gymnasium-Robotics by pip install and the version is 1.2.0, but why the corresponding code snippet is different?

[Proposal] Make mujoco_util getter functions return copy of values

Proposal

Right now, get_joint_qpos and get_joint_qvel return views from the underlying qpos and qvel of the mujoco object.
This can be potentially dangerous and a source of bugs.

Gymnasium-Robotics/gymnasium_robotics/utils/mujoco_utils.py

Line 215 in 215911a

return data.qvel[start_idx:end_idx]

If the users are unaware of this and modify the outputs of these functions, they will unknowingly modify the underlying qpos and qvel of the simulation.

Indeed, there is already unintended (but luckily safe) behavior in the codebase:

Gymnasium-Robotics/gymnasium_robotics/envs/fetch/fetch_env.py

Lines 389 to 396 in 215911a

 object_qpos = self._utils.get_joint_qpos( 

 self.model, self.data, "object0:joint" 

 ) 

 assert object_qpos.shape == (7,) 

 object_qpos[:2] = object_xpos 

 self._utils.set_joint_qpos( 

 self.model, self.data, "object0:joint", object_qpos 

 )

After executing line 393, if we inspect that values of self.data.qpos, we will see that they have already been changed to object_xpos. This is probably not intended.

Luckily, that change in self.data.qpos is exactly what we want to set in the next line, which ends up overwriting the same indexes in self.data.qpos with object_expos so there is no bug.

Pitch

We could just return a copy of the views by wrapping them in np.array calls like so:

  return np.array(data.qvel[start_idx:end_idx])

Checklist

I have checked that there is no similar issue in the repo (required)

[Question] Why remove IKController step mode from FrankaKitchen-v1?

Hi!

I noticed that the IKController step mode in v1.2.0 was removed in v1.2.1.

Also, from the release note, I found related descriptions:

In addition, the updates made for the first release of FrankaKitchen-v1 environment have been reverted in order for the environment to resemble more its original version in relay-policy-learning and D4RL

I can't quite understand the causal relationship between the above description and the removal of the IKController?

Is it because IKController causes differences in the training process compared to the original?

Could you give a more detailed explanation?

Thanks in advance!

[Question] Observations meaning

Hello!

I would like to ask what do the observations in environments mean specifically? I'm especially interested in 'HandReach-v0' and there are 63 observables, however I could not find and information about their meaning.

For example, the observations for OpenAI Gym are described here:
https://github.com/openai/gym/wiki/CartPole-v0

[Bug Report] Maze/AntMaze issues

Hi, I believe I have found a couple of issues in the Maze/AntMaze environments. I have resolved both of these issues in commit 5573d5e, and I’m happy to submit a PR.

1) AntMaze sparse reward always zero

For continuing tasks in AntMazeEnv, the sparse reward is always zero. This is because at each step, .compute_terminated() resets the goal when the Ant is sufficiently close, before the reward is calculated.

Here’s a code example to test this:

import numpy as np
import gymnasium as gym

env = gym.make("AntMaze_UMaze-v3", continuing_task=True, reward_type="sparse")

total_reward = 0

# 1000 "episodes" of 100 steps
for _ in range(1000):
    env.reset()

    for _ in range(100):
        action = env.action_space.sample()
        _, rew, _, _, _ = env.step(action)
        total_reward += rew

print(total_reward)

Currently this returns exactly zero since no reward is collected, but placing .compute_reward() above .compute_terminated() gives a non-zero reward. In PointMaze, this issue was fixed in commit ace181e.

2) AntMaze can reset into a terminal state

The Ant will sometimes start within the goal radius. This is because there is a maze_size_scaling factor missing in the distance check in MazeEnv.generate_reset_pos().

In AntMaze maze_size_scaling = 4, so the xy position noise can be up to 1.0 in each direction. An unfortunate combination of goal noise and reset noise can cause the Ant to start within 0.45 of self.goal. This issue does not affect PointMaze because there maze_size_scaling = 1.

Here’s a code example to test this:

import numpy as np
import gymnasium as gym

env = gym.make("AntMaze_UMaze-v3", continuing_task=True)

for _ in range(1000):
    obs, _ = env.reset()
    dist = np.linalg.norm(obs["achieved_goal"] - obs["desired_goal"])
    assert dist > 0.45

Checklist

I have checked that there is no similar issue in the repo (required)

[Question] TypeError: 'NoneType' object is not callable in Rendering Fetch Environments

Hi,

I'm getting this error at the end of rendering the Fetch environments https://github.com/Farama-Foundation/Gymnasium-Robotics in "human" mode:

Exception ignored in: <function WindowViewer.del at 0x7fb4ea7d2680>
Traceback (most recent call last):
File "miniconda/envs/gym/lib/python3.7/site-packages/gymnasium/envs/mujoco/mujoco_rendering.py", line 335, in del
File "miniconda/envs/gym/lib/python3.7/site-packages/gymnasium/envs/mujoco/mujoco_rendering.py", line 328, in free
TypeError: 'NoneType' object is not callable

I'm able to render the entire scene but when the program closes I get this error. Would appreciate more insight into whether it is a glfw based issue, mujoco environment based issue, or another package, thanks!

[Bug Report] Minor User Warning : WARN: Overriding environment HandManipulateBlockRotateZ-v1 already in registry.

Describe the bug
Getting user warning UserWarning: WARN: Overriding environment HandManipulateBlockRotateZ-v1 already in registry.
when importing Gym.
Code example

System Info
Describe the characteristic of your environment:
gym 0.26
python 3.10
windows

Python version

Checklist

Updating the init file will fix it.

[Bug Report] AntEnv obs[:2] is not x, y but z coordinate and x-orientation

Describe the bug
The _get_obs function does the following transformation to the observation space
achieved_goal = ant_obs[:2]
observation = ant_obs[2:]
But reading AntEnv V4 the pos 0 and pos 1 observations are not the final x and y coordinates. As the antenv is directly queried and no computation is done in the environment it's clear that this is an error.

Checklist

I have checked that there is no similar issue in the repo (required)

[Bug Report] False information drawn from MuJoCo simulator while using the Gymnasium-Robotics API's MuJoCo utility functions for custom FetchPObstaclePickAndPlace environment.

Hello dear members of the Farama Team!

I have already address this as an issue in #183 and creating this new issue as a kind reminder.
I have made a fork of the Gymnasium-Robotics API. I am working on creating a custom environment, based on the 'FetchPickAndPlace-v2' environment. This new environment will introduce an obstacle in the simulation. I have created the appropriate .xml file that introduces the obstacle, which is named 'objstacle0'. For this environment I wanted to add to the original observation of the 'FetchPickAnPlace' environment the following:

The position of the obstacle in the 3D space.
The relative position of the robot gripper and the obstacle.
The relative position of the object and the obstacle.

However, after creating the environment and performing some steps in it, the environment seems to return 0 values for the new three values added to the observation. An example can be seen in the picture bellow:

To further debug this behaviour I imported mujoco and I made the mujoco_utils from hidden to public attribute of the environment, so I can access the low level methods that extract the data from the MuJoCo simulator. I was surprised to see that the obstacle and the object where returning the same positions:

I retrieved this data by using the 'get_site_xpos()' method of the 'mujoco_utils' package. However, that is definitely not the case because from the observation we can see that the achieved goal has different value. Furthermore you can see from the image of the simulation that the object is in different position to the one of the obstacle.

Here is the overwritten version of the 'generate_mujoco_observations()' method that I am using:

def generate_mujoco_observations(self):
# positions
grip_pos = self._utils.get_site_xpos(self.model, self.data, "robot0:grip")

    dt = self.n_substeps * self.model.opt.timestep
    grip_velp = (
        self._utils.get_site_xvelp(self.model, self.data, "robot0:grip") * dt
    )

    robot_qpos, robot_qvel = self._utils.robot_get_obs(
        self.model, self.data, self._model_names.joint_names
    )
    if self.has_object:
        object_pos = self._utils.get_site_xpos(self.model, self.data, "object0")
        # rotations
        object_rot = rotations.mat2euler(
            self._utils.get_site_xmat(self.model, self.data, "object0")
        )
        # velocities
        object_velp = (
            self._utils.get_site_xvelp(self.model, self.data, "object0") * dt
        )
        object_velr = (
            self._utils.get_site_xvelr(self.model, self.data, "object0") * dt
        )
        # gripper state
        object_rel_pos = object_pos - grip_pos
        object_velp -= grip_velp
    else:
        object_pos = (
            object_rot
        ) = object_velp = object_velr = object_rel_pos = np.zeros(0)
    gripper_state = robot_qpos[-2:]

    gripper_vel = (
        robot_qvel[-2:] * dt
    )  # change to a scalar if the gripper is made symmetric
    
    # Extract the positions of the gripper and the object
    #grip_pos = self._utils.get_site_xpos(self.model, self.data, "robot0:grip")
    #object_pos = self._utils.get_site_xpos(self.model, self.data, "object0")
    
    # Calculate the obstacle's position (assuming its site name is "obstacle")
    obstacle_pos = self._utils.get_site_xpos(self.model, self.data, "obstacle0")
    
    # Calculate the relative positions
    gripper_to_obstacle = obstacle_pos - grip_pos
    object_to_obstacle = obstacle_pos - object_pos
    
    # Extend the original observations with the relative positions
    #extended_observations = observations + (gripper_to_obstacle,) + (object_to_obstacle,)        
    return (
        grip_pos,
        object_pos,
        object_rel_pos,
        gripper_state,
        object_rot,
        object_velp,
        object_velr,
        grip_velp,
        gripper_vel,
        obstacle_pos,
        gripper_to_obstacle,
        object_to_obstacle
    )

And here the code of the obstacle_pick_and_place.xml:

<include file="shared.xml" />

<worldbody>
	<geom name="floor0" pos="0.8 0.75 0" size="0.85 0.7 1" type="plane" condim="3" material="floor_mat" />
	<body name="floor0" pos="0.8 0.75 0">
		<site name="target0" pos="0 0 0.5" size="0.02 0.02 0.02" rgba="1 0 0 1" type="sphere" />
	</body>

	<include file="robot.xml" />
	
	<body pos="1.3 0.75 0.2" name="table0">
		<geom size="0.25 0.35 0.2" type="box" mass="2000" material="table_mat" />
	</body>

	<body pos="1.3 0.75 0.48" name="obstacle0">
		<geom  size="0.025 0.35 0.075" type="box" rgba="0.6 0.3 0 1" />
	</body>
	
	<body name="object0" pos="0.025 0.025 0.025">
		<joint name="object0:joint" type="free" damping="0.01" />
		<geom size="0.025 0.025 0.025" type="box" condim="3" name="object0" material="block_mat" mass="2" />
		<site name="object0" pos="0 0 0" size="0.02 0.02 0.02" rgba="1 0 0 1" type="sphere" />
	</body>

	<light directional="true" ambient="0.2 0.2 0.2" diffuse="0.8 0.8 0.8" specular="0.3 0.3 0.3" castshadow="false" pos="0 0 4" dir="0 0 -1" name="light0" />
</worldbody>

<actuator>
	<position ctrllimited="true" ctrlrange="0 0.2" joint="robot0:l_gripper_finger_joint" kp="30000" name="robot0:l_gripper_finger_joint" user="1" />
	<position ctrllimited="true" ctrlrange="0 0.2" joint="robot0:r_gripper_finger_joint" kp="30000" name="robot0:r_gripper_finger_joint" user="1" />
</actuator>

Could you please help me debug and fix this behaviour ?

Thank you very much in advance for all the valuable help and support !!!

Kind regards,

Christos Peridis

[Proposal] Modify the signature of the method GoalEnv.reset() to match the signature of the base method

Proposal

In core.py, the signature:

def reset(self, seed: Optional[int] = None):

could be changed to:

def reset(self, options=None, seed: Optional[int] = None, infos=None):

to match the new signature of resets in gym (version 0.22).

[Bug Report] double registration import issue

There are issues with importing gymnasium_robotics/__init__.py twice

$ pip list | grep gym           
gymnasium                     0.27.0

when you import gymansium_robotics it appers that /__init__.py is called twice

Git version

~/Gymnasium-Robotics]$ git pull           
Already up to date

~/Gymnasium-Robotics]$ pip install . && py
Defaulting to user installation because normal site-packages is not writeable
Processing /home/master-andreas/Gymnasium-Robotics
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: numpy<1.24.0,>=1.21.0 in /usr/lib/python3.10/site-packages (from gymnasium-robotics==1.1.0) (1.23.5)
Requirement already satisfied: mujoco>=2.3.1.post1 in /home/master-andreas/.local/lib/python3/site-packages (from gymnasium-robotics==1.1.0) (2.3.1.post1)
Requirement already satisfied: gymnasium>=0.26 in /home/master-andreas/.local/lib/python3/site-packages (from gymnasium-robotics==1.1.0) (0.27.0)
Requirement already satisfied: jax-jumpy>=0.2.0 in /home/master-andreas/.local/lib/python3/site-packages (from gymnasium>=0.26->gymnasium-robotics==1.1.0) (0.2.0)
Requirement already satisfied: typing-extensions>=4.3.0 in /usr/lib/python3.10/site-packages (from gymnasium>=0.26->gymnasium-robotics==1.1.0) (4.4.0)
Requirement already satisfied: shimmy<1.0,>=0.1.0 in /home/master-andreas/.local/lib/python3/site-packages (from gymnasium>=0.26->gymnasium-robotics==1.1.0) (0.2.0)
Requirement already satisfied: cloudpickle>=1.2.0 in /home/master-andreas/.local/lib/python3/site-packages (from gymnasium>=0.26->gymnasium-robotics==1.1.0) (2.2.0)
Requirement already satisfied: gymnasium-notices>=0.0.1 in /home/master-andreas/.local/lib/python3/site-packages (from gymnasium>=0.26->gymnasium-robotics==1.1.0) (0.0.1)
Requirement already satisfied: glfw in /home/master-andreas/.local/lib/python3/site-packages (from mujoco>=2.3.1.post1->gymnasium-robotics==1.1.0) (2.5.5)
Requirement already satisfied: absl-py in /home/master-andreas/.local/lib/python3/site-packages (from mujoco>=2.3.1.post1->gymnasium-robotics==1.1.0) (1.3.0)
Requirement already satisfied: pyopengl in /home/master-andreas/.local/lib/python3/site-packages (from mujoco>=2.3.1.post1->gymnasium-robotics==1.1.0) (3.1.6)
Building wheels for collected packages: gymnasium-robotics
  Building wheel for gymnasium-robotics (pyproject.toml) ... done
  Created wheel for gymnasium-robotics: filename=gymnasium_robotics-1.1.0-py3-none-any.whl size=1485120 sha256=79a9063e315f99716b0f7ff51ba6ceb5e7d82534aea3f4b88936593645b42db1
  Stored in directory: /home/master-andreas/.cache/pip/wheels/fb/22/bd/db3af103b5ded132ca76abe0558b08910f496de990b605d1d8
Successfully built gymnasium-robotics
Installing collected packages: gymnasium-robotics
  Attempting uninstall: gymnasium-robotics
    Found existing installation: gymnasium-robotics 1.1.0
    Uninstalling gymnasium-robotics-1.1.0:
      Successfully uninstalled gymnasium-robotics-1.1.0
Successfully installed gymnasium-robotics-1.1.0
Python 3.10.8 (main, Nov  1 2022, 14:18:21) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import gymnasium_robotics
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment FetchSlide-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment FetchSlide-v2 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment FetchPickAndPlace-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment FetchPickAndPlace-v2 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment FetchReach-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment FetchReach-v2 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment FetchReach-v3 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment FetchPush-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment FetchPush-v2 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandReach-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandReach-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateZ-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateZ-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateZ_BooleanTouchSensors-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateZ_BooleanTouchSensors-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateZ_ContinuousTouchSensors-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateZ_ContinuousTouchSensors-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateParallel-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateParallel-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateParallel_BooleanTouchSensors-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateParallel_BooleanTouchSensors-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateParallel_ContinuousTouchSensors-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateParallel_ContinuousTouchSensors-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateXYZ-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateXYZ-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateXYZ_BooleanTouchSensors-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateXYZ_BooleanTouchSensors-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateXYZ_ContinuousTouchSensors-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateXYZ_ContinuousTouchSensors-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockFull-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockFull-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlock-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlock-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlock_BooleanTouchSensors-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlock_BooleanTouchSensors-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlock_ContinuousTouchSensors-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlock_ContinuousTouchSensors-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateEggRotate-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateEggRotate-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateEggRotate_BooleanTouchSensors-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateEggRotate_BooleanTouchSensors-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateEggRotate_ContinuousTouchSensors-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateEggRotate_ContinuousTouchSensors-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateEggFull-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateEggFull-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateEgg-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateEgg-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateEgg_BooleanTouchSensors-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateEgg_BooleanTouchSensors-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateEgg_ContinuousTouchSensors-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateEgg_ContinuousTouchSensors-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulatePenRotate-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulatePenRotate-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulatePenRotate_BooleanTouchSensors-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulatePenRotate_BooleanTouchSensors-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulatePenRotate_ContinuousTouchSensors-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulatePenRotate_ContinuousTouchSensors-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulatePenFull-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulatePenFull-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulatePen-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulatePen-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulatePen_BooleanTouchSensors-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulatePen_BooleanTouchSensors-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulatePen_ContinuousTouchSensors-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulatePen_ContinuousTouchSensors-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment FetchSlideDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment FetchSlideDense-v2 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment FetchPickAndPlaceDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment FetchPickAndPlaceDense-v2 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment FetchReachDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment FetchReachDense-v2 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment FetchReachDense-v3 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment FetchPushDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment FetchPushDense-v2 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandReachDense-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandReachDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateZDense-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateZDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateZ_BooleanTouchSensorsDense-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateZ_BooleanTouchSensorsDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateZ_ContinuousTouchSensorsDense-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateZ_ContinuousTouchSensorsDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateParallelDense-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateParallelDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateParallel_BooleanTouchSensorsDense-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateParallel_BooleanTouchSensorsDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateParallel_ContinuousTouchSensorsDense-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateParallel_ContinuousTouchSensorsDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateXYZDense-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateXYZDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateXYZ_BooleanTouchSensorsDense-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateXYZ_BooleanTouchSensorsDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateXYZ_ContinuousTouchSensorsDense-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockRotateXYZ_ContinuousTouchSensorsDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockFullDense-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockFullDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockDense-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlockDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlock_BooleanTouchSensorsDense-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlock_BooleanTouchSensorsDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlock_ContinuousTouchSensorsDense-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateBlock_ContinuousTouchSensorsDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateEggRotateDense-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateEggRotateDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateEggRotate_BooleanTouchSensorsDense-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateEggRotate_BooleanTouchSensorsDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateEggRotate_ContinuousTouchSensorsDense-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateEggRotate_ContinuousTouchSensorsDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateEggFullDense-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateEggFullDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateEggDense-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateEggDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateEgg_BooleanTouchSensorsDense-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateEgg_BooleanTouchSensorsDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateEgg_ContinuousTouchSensorsDense-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulateEgg_ContinuousTouchSensorsDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulatePenRotateDense-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulatePenRotateDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulatePenRotate_BooleanTouchSensorsDense-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulatePenRotate_BooleanTouchSensorsDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulatePenRotate_ContinuousTouchSensorsDense-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulatePenRotate_ContinuousTouchSensorsDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulatePenFullDense-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulatePenFullDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulatePenDense-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulatePenDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulatePen_BooleanTouchSensorsDense-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulatePen_BooleanTouchSensorsDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulatePen_ContinuousTouchSensorsDense-v0 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py:520: UserWarning: WARN: Overriding environment HandManipulatePen_ContinuousTouchSensorsDense-v1 already in registry.
  logger.warn(f"Overriding environment {new_spec.id} already in registry.")
>>>

pip version

...
$ pip uninstall gymnasium_robotics
...
$ pip install gymnasium_robotics  
...
[master-andreas@master-pc ~]$ py                            
Python 3.10.8 (main, Nov  1 2022, 14:18:21) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import gymnasium_robotics
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium_robotics/__init__.py", line 1, in <module>
    from gymnasium.envs.registration import register
  File "/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/__init__.py", line 12, in <module>
    from gymnasium.envs.registration import make, spec, register, registry, pprint_registry
  File "/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/__init__.py", line 352, in <module>
    load_env_plugins()
  File "/home/master-andreas/.local/lib/python3.10/site-packages/gymnasium/envs/registration.py", line 309, in load_env_plugins
    fn = plugin.load()
  File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 173, in load
    return functools.reduce(getattr, attrs, module)
AttributeError: partially initialized module 'gymnasium_robotics' has no attribute 'register_robotics_envs' (most likely due to a circular import)
>>>

[Bug Report] pip mamujoco install error

Describe the bug
I tried installing the mamujoco environement and pip reports that the extra "mamujoco" is not provided by gymnasium-robotics

Code example
pip install gymnasium-robotics[mamujoco]

WARNING: gymnasium-robotics 1.2.0 does not provide the extra 'mamujoco'

System Info
Describe the characteristic of your environment:

gymnasium was installed with pip
OS Version: Ubuntu 22.04.1 LTS (Jammy Jellyfish)
Python 3.10

Additional context
Add any other context about the problem here.

Checklist

I have checked that there is no similar issue in the repo

[Bug Report] `pyright` generates error with `Gymnasium==0.27.0`

[ ~/Gymnasium-Robotics]$ pre-commit run --all-files     
black....................................................................Passed
codespell................................................................Passed
flake8...................................................................Passed
isort....................................................................Passed
pyupgrade................................................................Passed
pyright..................................................................Failed
- hook id: pyright
- exit code: 1

Configuration file not found at /home/master-andreas/Gymnasium-Robotics/pyproject.toml.
pyproject.toml file found at /home/master-andreas/Gymnasium-Robotics.
Loading pyproject.toml file at /home/master-andreas/Gymnasium-Robotics/pyproject.toml
Assuming Python platform Linux
typeshedPath /home/master-andreas/Gymnasium-Robotics/typeshed is not a valid directory.
stubPath /home/master-andreas/Gymnasium-Robotics/typings is not a valid directory.
Searching for source files
Found 27 source files
pyright 1.1.283
/home/master-andreas/Gymnasium-Robotics/tests/utils.py
  /home/master-andreas/Gymnasium-Robotics/tests/utils.py:7:30 - error: "registry" is not exported from module "gymnasium.envs" (reportPrivateImportUsage)
1 error, 0 warnings, 0 informations 
Completed in 4.34sec

`env.render(mode='rgb_array')` returns `none`

I’m trying to run MaMuJoCo on a headless server, but when I use env.render(mode='rgb_array'), it returns none. I don’t understand why.
Could you please help me? Thank you!

[Question] Abnormal franka arm swing when using mocap

Hi!

I'm trying to use mocap to control the franka arm like fetch_pick_and_place.env.

After I created this environment and tested it, I noticed that the arm drifted abnormally each time I initiated this env, even though I gave the initial configuration for data.qpos and the initial pose for mocap.

Here is a video to illustrate my dilemma

swing.mp4

in which I repeat the startup of this env.

Currently, I think the triggering reason might be that the given initial configuration is different from that solved by the built-in inverse kinematics algorithm. For comparison, I also initialized fetch_pick_and_place.env in the same way and did not notice a similar swing. If the key factor is the inverse kinematics algorithm, why can the fetch arm maintain almost the same configuration at each initial stage? The fetch arm is also redundant.

Here is the video to describe the loop to initialize fetch_pick_and_place.env

fetch.mp4

Also, maybe I missed some other factors.

Could you give me some advice on how to solve it?

Thanks in advance!

[Question] Maze Dense Reward

Question

Looking at the dense reward function for Maze Env:

return np.exp(-np.linalg.norm(desired_goal - achieved_goal))

The agent seems to prefer sitting the ball as close as possible to the goal without touching it after optimisation.

This makes sense given there is no bonus for reaching the reward and the reward is positive for all time steps.

Why is the dense reward formulated this way?

[Bug Report] Gym 0.26 compatibility

I believe this is not yet compatible with the latest gym (0.26.0)

I tried the following code with and without unwrapped, but render consistently returns None.

env = EnvCompatibility(gym.make("FetchReach-v1").unwrapped, render_mode="rgb_array")

Should we consider migrating to the new API if it's going to last?

[Question] Why is gym master required?

Hello,
I was wondering why is gym master required and not a more specific version like gym >= 0.22 for instance?
This prevents other projects from limiting the gym robotics version in case there are breaking changes in newer gym versions.

[Question] Wall Contact in PointMaze

I am working with the PointMaze environments from D4RL. I noticed that the point can partially go through walls. In addition, the point has slightly emerged under the floor. Is that intentional?

[Question] `reset_mocapxbody()` in `mocap_set_action()`

Question

Hi, I have a robotic arm and am controlling it using a mocap body and mocap_set_actoin() provided in mujoco_utils. What I am observing is that every time I call mocap_set_action, reset_mocapxbody() is called inside it which leads to an undesirable movement of the end-effector as it is trying to change the mocap body position. I am wondering if reset_mocapxbody is necessary for mocap_set_action. I understand the purpose of this function but it causes an extra unwanted movement of the end-effector.
Thanks.

[Question] How can I change the distance_threshold in fetch task?

Question

I want to change the distance_threshold value to improve the difficulty of the fetch task.
Like handenv, I can use env = gym.make('HandReach-v0',distance_threshold=0.001), but env = gym.make('FetchReach-v1',distance_threshold=0.001) doesn't work.
Then I try to use env.distance_threshold=0.001 to set the variable value, but the compute_reward function and the info['is_success'] returned by env.step function seem to use the default distance_threshold 0.05. So what should I do if I want to change the distance_threshold in fetch task?

[Bug Report] HTML Doc Generation gives warnings

[~/Gymnasium-Robotics]$ cd docs              
[~/Gymnasium-Robotics/docs]$ make dirhtml _build
Running Sphinx v5.2.3
loading pickled environment... done
myst v0.18.1: MdParserConfig(commonmark_only=False, gfm_only=False, enable_extensions=[], disable_syntax=[], all_links_external=False, url_schemes=('http', 'https', 'mailto', 'ftp'), ref_domains=None, highlight_code_blocks=True, number_code_blocks=[], title_to_header=False, heading_anchors=None, heading_slug_func=None, footnote_transition=True, words_per_minute=200, sub_delimiters=('{', '}'), linkify_fuzzy_links=True, dmath_allow_labels=True, dmath_allow_space=True, dmath_allow_digits=True, dmath_double_inline=False, update_mathjax=True, mathjax_classes='tex2jax_process|mathjax_process|math|output_area')
building [mo]: targets for 0 po files that are out of date
building [dirhtml]: targets for 0 source files that are out of date
updating environment: 0 added, 3 changed, 0 removed
reading sources... [100%] envs/hand_touch/index                                  
/home/master-andreas/Gymnasium-Robotics/docs/envs/fetch/index.md:6: WARNING: Document headings start at H2, not H1 [myst.header]
/home/master-andreas/Gymnasium-Robotics/docs/envs/fetch/index.md:15: CRITICAL: Problems with "raw" directive path:
InputError: [Errno 2] No such file or directory: 'envs/fetch/list.html'.
/home/master-andreas/Gymnasium-Robotics/docs/envs/fetch/index.md:19: WARNING: toctree contains reference to nonexisting document 'envs/fetch/FetchReach'
/home/master-andreas/Gymnasium-Robotics/docs/envs/fetch/index.md:19: WARNING: toctree contains reference to nonexisting document 'envs/fetch/FetchSlide'
/home/master-andreas/Gymnasium-Robotics/docs/envs/fetch/index.md:19: WARNING: toctree contains reference to nonexisting document 'envs/fetch/FetchPickAndPlace'
/home/master-andreas/Gymnasium-Robotics/docs/envs/fetch/index.md:19: WARNING: toctree contains reference to nonexisting document 'envs/fetch/FetchPush'
/home/master-andreas/Gymnasium-Robotics/docs/envs/hand/index.md:6: WARNING: Document headings start at H2, not H1 [myst.header]
/home/master-andreas/Gymnasium-Robotics/docs/envs/hand/index.md:15: CRITICAL: Problems with "raw" directive path:
InputError: [Errno 2] No such file or directory: 'envs/hand/list.html'.
/home/master-andreas/Gymnasium-Robotics/docs/envs/hand/index.md:19: WARNING: toctree contains reference to nonexisting document 'envs/hand/HandReach'
/home/master-andreas/Gymnasium-Robotics/docs/envs/hand/index.md:19: WARNING: toctree contains reference to nonexisting document 'envs/hand/HandBlock'
/home/master-andreas/Gymnasium-Robotics/docs/envs/hand/index.md:19: WARNING: toctree contains reference to nonexisting document 'envs/hand/HandEgg'
/home/master-andreas/Gymnasium-Robotics/docs/envs/hand/index.md:19: WARNING: toctree contains reference to nonexisting document 'envs/hand/HandPen'
/home/master-andreas/Gymnasium-Robotics/docs/envs/hand_touch/index.md:20: CRITICAL: Problems with "raw" directive path:
InputError: [Errno 2] No such file or directory: 'envs/hand_touch/list.html'.
/home/master-andreas/Gymnasium-Robotics/docs/envs/hand_touch/index.md:24: WARNING: toctree contains reference to nonexisting document 'envs/hand_touch/HandBlockTouchSensors'
/home/master-andreas/Gymnasium-Robotics/docs/envs/hand_touch/index.md:24: WARNING: toctree contains reference to nonexisting document 'envs/hand_touch/HandEggTouchSensors'
/home/master-andreas/Gymnasium-Robotics/docs/envs/hand_touch/index.md:24: WARNING: toctree contains reference to nonexisting document 'envs/hand_touch/HandPenTouchSensors'
looking for now-outdated files... none found
pickling environment... done
checking consistency... /home/master-andreas/Gymnasium-Robotics/docs/404.md: WARNING: document isn't included in any toctree
/home/master-andreas/Gymnasium-Robotics/docs/README.md: WARNING: document isn't included in any toctree
done
preparing documents... done
writing output... [100%] index                                                   
generating indices... genindex done
writing additional pages... search done
copying images... [100%] _static/videos/fetch/FetchPickAndPlace.gif              
copying static files... done
copying extra files... done
dumping search index in English (code: en)... done
dumping object inventory... done
build succeeded, 18 warnings.

The HTML pages are in _build/dirhtml.
Running Sphinx v5.2.3

Sphinx error:
Builder name _build not registered or available through entry point
make: *** [Makefile:20: _build] Error 2

[Proposal] Add pydocstyle to pre-commit

Pre-commit is missing pydocstyle check which is helpful for ensuring high quality documented code

[Bug Report] The `tasks_to_complete` variable does not reset properly in `FrankaKitchen-v1`

Describe the bug
The tasks_to_complete variable does not reset properly in FrankaKitchen-v1. The info['tasks_to_complete'] returned from the reset() function does match the tasks_to_complete argument passed when initializing the environment. However, in subsequent calls to the step() function, the info['tasks_to_complete'] excludes tasks removed in the previous episode. Also, no reward will be generated from the removed tasks.

Code example

env = gym.make('FrankaKitchen-v1', tasks_to_complete=['microwave', 'kettle'])

After creating the environment, complete the kettle task in an episode and then execute the following code:

obs, info = env.reset()
print(info['tasks_to_complete'])
obs, reward, terminated, truncated, info = env.step(env.action_space.sample())
print(info['tasks_to_complete'], info['episode_task_completions'])

This will output:

{'kettle', 'microwave'}
['microwave'] []

Solution
Change task_to_complete to tasks_to_complete in reset function of kitchen_env:

        self.tasks_to_complete = set(self.goal.keys())
        info = {
            "tasks_to_complete": self.tasks_to_complete,
            "episode_task_completions": [],
            "step_task_completions": [],
        }

System Info

pip install within a conda env
Ubuntu 20.04
Python 3.8

Checklist

I have checked that there is no similar issue in the repo (required)

[Bug Report] Rendering problem after first truncation if env is wrapped with RecordVideo

Describe the bug

If I wrap the environment with RecordVideo wrapper when using rgb_array/rgb_array_list as the render_mode, env.render() returns arrays with zero values after the first time the environment is truncated. Reseting the environment does not fix the rendering output. This does not happen when not using the RecordVideo wrapper on Gymnasium-Robotics environments, or when using RecordVideo wrapper on the basic Gymnasium environments (e.g., classic control).

Code example

import gymnasium as gym
from gymnasium.wrappers import RecordVideo

env = gym.make("HandReach-v1", render_mode="rgb_array",max_episode_steps=5)
env = RecordVideo(env, video_folder="videos/",disable_logger=True)

observation, info = env.reset()
for i in range(7):
    action = env.action_space.sample() 
    observation, reward, terminated, truncated, info = env.step(action)

    if terminated or truncated:
        observation, info = env.reset()
        print("reset")
    else:
        out = env.render()
        print(out.max())

env.close()

System Info

macOS 13.1
python 3.10.8
gymnasium 0.27.0 (pypi)
gymnasium-robotics 1.2.0 (pypi)
moviepy 1.0.3 (pypi)
ffmpeg-python 0.2.0 (pypi)

Additional context

Checklist

I have checked that there is no similar issue in the repo (required)

[Proposal] Environment should terminate when adroit hand pen drops the pen

Proposal

In AdroitHandPen, when the agent drops the pen, there is no way to recover, but the environment still does not terminate. The proposal, as in #111, is to enable environment termination on pen drop.

[Bug] ImportError on MujocoRenderer from Gymnasium.

If you are submitting a bug report, please fill in the following details and use the tag [bug].

Getting an ImportError in gymnasium_robotics/envs/robot_env.py, line 282. Error: ImportError: cannot import name 'MujocoRenderer' from 'gymnasium.envs.mujoco.mujoco_rendering.py

Code example
import gymnasium as gym
env = gym.make("FetchPushDense-v2")

System Info
Describe the characteristic of your environment:
Latest gymnasium and gymnasium-robotics by pip. Mujoco-2.3.2 tested with both source and pip

Ubuntu 18.04

Tested with python 3.7 and 3.10.9

Additional context
Might be a issue in Gymnasium and not Gymnasium-Robotics.

Checklist

I have checked that there is no similar issue in the repo (required)

[Proposal] Mujoco-v5

Proposal 0

After the Gymnasium/envs/mujoco gets moved here, create a new revision (v5), that fixes the 0 const observations in the following domains:

Humanoid (+standup): Farama-Foundation/Gymnasium#204, Farama-Foundation/Gymnasium#504
Ant: Farama-Foundation/Gymnasium#204, Farama-Foundation/Gymnasium#214
Reacher: Farama-Foundation/Gymnasium#220
InvertedDoublePendulum: Farama-Foundation/Gymnasium#228, Farama-Foundation/Gymnasium#500
InvertedPendulum: Farama-Foundation/Gymnasium#500

Note: I have written a quick test, that only fails in the above environments

_MUJOCO_GYM_ENVIROMENTS = [  # Note: this could be made dynamic, i was just too lazy to do it
    "Ant-v4",
    "HalfCheetah-v4",
    "Hopper-v4",
    "HumanoidStandup-v4",
    "Humanoid-v4",
    "Reacher-v4",
    "Swimmer-v4",
    "Pusher-v4",
    "Walker2d-v4",
    "InvertedPendulum-v4",
    "InvertedDoublePendulum-v4",
]

@pytest.mark.parametrize("env_name", _MUJOCO_GYM_ENVIROMENTS)
def test_zeros_in_observation(env_name: str):
    """Asserts that all enviroments containt valid (non-zero) observations

    This Tests was created because some mujoco_enviroments pre version 5 did have part of the observation space that was
    """
    if env_name == "Ant-v4":
        env = gym.make(env_name, use_contact_forces=True)
    else:
        env = gym.make(env_name)
    env.action_space.seed(0)
    env.reset(seed=0)
    observation, _, _, _, _ = env.step(env.action_space.sample())
    assert (observation != 0).all()

Proposal 1

Register some new environments to the suite from mamujoco like coupled half cheetah https://github.com/Kallinteris-Andreas/Gymnasium-Robotics-Kalli/blob/main/gymnasium_robotics/envs/multiagent_mujoco/coupled_half_cheetah.py#L27

Proposal 2

Update Hopper and Walker2D models to not require coordinate="Global": google-deepmind/mujoco#833

And fix Walker2D feet Farama-Foundation/Gymnasium#477

Checklist

I have checked that there is no similar issue in the repo (required)

[Proposal] Add `terminate_when_unhealthy` option to mamujoco

Proposal

Would it be possible to add the terminate_when_unhealthy option from the single agent version in Gymnasium to mamujoco?

Motivation

For some experiments I would like to keep the agent running and even when the state becomes unhealthy.

Pitch

Add an option to pass the terminate_when_unhealthy option to the underlying single agent environment through with the mamujoco parallel env creation.

Checklist

I have checked that there is no similar issue in the repo

[Question] pip install gymnasium-robotics[mujoco-py]

Hi!

When I use this command pip install gymnasium-robotics[mujoco-py],

no matches found: gymnasium-robotics[mujoco-py] is displayed.

Is this installation correct and still available?

Thanks!

[Question] How can I fix the orientation (just like FetchEnv) of Franka Panda in Franka kitchen?

Question

Hi, thanks for your nice work.
I would like to ask how can I fix the orientation (just like FetchEnv) of Franka Panda in Franka kitchen.
I have try to set the action[3:7]=[0, 0, 0] of Franka Panda, but when I render the environment I see the orientation still changed. I also print the achieved pose of the EEF site in step() after:

for _ in range(self.control_steps):
                delta_qpos = self.controller.compute_qpos_delta(
                    target_eef_pose, target_orientation
                )
                ctrl_action[:7] = self.data.ctrl.copy()[:7] + delta_qpos[:7]

                # Do not use `do_simulation`` method from MujocoEnv: value error due to discrepancy between
                # the action space and the simulation control input when using IK controller.
                # TODO: eliminate error check in MujocoEnv (action space can be different from simulaton control input).
                self.data.ctrl[:] = ctrl_action
                mujoco.mj_step(self.model, self.data, nstep=self.frame_skip)

                if self.render_mode == "human":
                    self.render()

I see the orientation indeed changed.
I know mocap may help me in this task (need to modify the xml file), but I really want to know why the orientation changed. Is it because of the IK controller?
By the way, what is the meaning of self.control_steps.

[Bug Report] AdroitHandRelocate set env state is unstable when using states where the ball is being grasped.

If you are submitting a bug report, please fill in the following details and use the tag [bug].

Describe the bug
It seems mujoco is a little unstable when it comes to setting state when active manipulation + contact is going on between the hand and a non-articulated actor like the sphere. (not an issue in AdroitDoor, and surprisingly not an issue it appears in AdroitPen).

In AdroitHandRelocate, setting env state to a state where the ball is grasped by the hand causes some instability and the ball will teleport out / penetrate the hand.

Code example
Take a succesfull trajectory from minari, replay it. Call env.get_env_state() to get the final state, then reset using the final state. The ball will usually teleport out. There is also another "bug" where the output of get_env_state includes 2 extraneous parts hand_qpos and palm_pos which are not part of the state space.

System Info
Ubuntu, Gymnasium 0.29, Python 3.9

Checklist

I have checked that there is no similar issue in the repo (required)

Solution?

I got around this by setting state twice. The first time I set I note the difference in ball position between where it actually got set to (after the one step of mujoco sim which self.set_state(qp, qv) does), and then add that difference to the state I wanted to set to

def set_env_state(self, state_dict):
        """
        Set the state which includes hand as well as objects and targets in the scene
        """
        assert self._state_space.contains(
            state_dict
        ), f"The state dictionary {state_dict} must be a member of {self._state_space}."
        qp = state_dict["qpos"]
        qv = state_dict["qvel"]

        self.model.body_pos[self.obj_body_id] = state_dict["obj_pos"]
        self.model.site_pos[self.target_obj_site_id] = state_dict["target_pos"]
        self.set_state(qp, qv)
        diff = self.model.body_pos[self.obj_body_id] - self.data.xpos[self.obj_body_id]
        # print(diff, "desired", self.model.body_pos[self.obj_body_id], "achieved", self.data.xpos[self.obj_body_id])  
        self.model.body_pos[self.obj_body_id] = state_dict["obj_pos"] + diff
        self.set_state(qp, qv)
        # print(diff, "desired", self.model.body_pos[self.obj_body_id], "achieved", self.data.xpos[self.obj_body_id])

[Bug Report] FetchReach-v2 render bug

The FetchReach-v2 crashes when initialized with `render_mode="human" (this does not happen for the other three fetch environments):

import gymnasium as gym

env = gym.make("FetchReach-v2", render_mode="human")
env.reset()
terminated = False

while not terminated:
    obs, reward, terminated, truncated, info = env.step(env.action_space.sample())

results in

Traceback (most recent call last):
  File "/home/finn/PycharmProjects/icml-2023-code/algos/sql_v2/robotics-gym.py", line 4, in <module>
    env.reset()
  File "/home/finn/PycharmProjects/icml-2023-code/venv/lib/python3.9/site-packages/gymnasium/wrappers/time_limit.py", line 69, in reset
    return self.env.reset(**kwargs)
  File "/home/finn/PycharmProjects/icml-2023-code/venv/lib/python3.9/site-packages/gymnasium/wrappers/order_enforcing.py", line 43, in reset
    return self.env.reset(**kwargs)
  File "/home/finn/PycharmProjects/icml-2023-code/venv/lib/python3.9/site-packages/gymnasium/wrappers/env_checker.py", line 45, in reset
    return env_reset_passive_checker(self.env, **kwargs)
  File "/home/finn/PycharmProjects/icml-2023-code/venv/lib/python3.9/site-packages/gymnasium/utils/passive_env_checker.py", line 208, in env_reset_passive_checker
    result = env.reset(**kwargs)
  File "/home/finn/PycharmProjects/icml-2023-code/venv/lib/python3.9/site-packages/gymnasium_robotics/envs/robot_env.py", line 184, in reset
    self.render()
  File "/home/finn/PycharmProjects/icml-2023-code/venv/lib/python3.9/site-packages/gymnasium_robotics/envs/robot_env.py", line 404, in render
    self._get_viewer(self.render_mode).render()
  File "/home/finn/PycharmProjects/icml-2023-code/venv/lib/python3.9/site-packages/gymnasium_robotics/envs/robot_env.py", line 427, in _get_viewer
    self._viewer_setup()
  File "/home/finn/PycharmProjects/icml-2023-code/venv/lib/python3.9/site-packages/gymnasium_robotics/envs/fetch/fetch_env.py", line 241, in _viewer_setup
    lookat = self.get_gripper_xpos()
AttributeError: 'MujocoPyFetchReachEnv' object has no attribute 'get_gripper_xpos'

Simply replacing lookat = self.get_gripper_xpos() with lookat = self._get_gripper_xpos() results in the environment executing as expected without error.

[Bug Report] error with 'pip install gym-robotics'

Describe the bug
After pip install gym (version 0.22.0), pip install gym-robotics leads to the following error:

Packages installed from PyPI cannot depend on packages which are not also hosted on PyPI.

And when installing directly gym-robotics with pip (without a prior installation of gym): pip install gym-robotics, the import of gym_robotics returns the following error:

File "<stdin>", line 1, in <module>
  File "/home/perrin/.conda/envs/p3/lib/python3.7/site-packages/gym_robotics/__init__.py", line 1, in <module>
    from gym.envs.registration import register
  File "/home/perrin/.conda/envs/p3/lib/python3.7/site-packages/gym/__init__.py", line 12, in <module>
    from gym.envs import make, spec, register
  File "/home/perrin/.conda/envs/p3/lib/python3.7/site-packages/gym/envs/__init__.py", line 10, in <module>
    _load_env_plugins()
  File "/home/perrin/.conda/envs/p3/lib/python3.7/site-packages/gym/envs/registration.py", line 725, in load_env_plugins
    fn = plugin.load()
  File "/home/perrin/.conda/envs/p3/lib/python3.7/site-packages/importlib_metadata/__init__.py", line 196, in load
    return functools.reduce(getattr, attrs, module)
AttributeError: module 'gym_robotics' has no attribute 'register_robotics_envs'

System Info
Ubuntu 22.04
Tested with Python 3.7.7 and 3.9.9

[Question] Is there a way to set the environment to a specific state in franka kitchen tasks?

I am trying to replay the expert demonstrations in the franka kitchen environment but I couldn't find any way of setting the environment to a specific state in the code. Any help would be appreciated!

[Proposal] Use mujoco bindings from deepmind instead of mujoco_py

Since mujoco_py is no longer maintained, and gym has also already updated its other mujoco envs to use deepmind's python bindings, gym-robotics needs to be updated as well.

[Bug Report] Temporary XML file is written to fixed path

Hi,

I would like to report a behavior of the Maze class that I believe to be a bug.

Describe the bug
The temporary XML file is always written to the same fixed path (on my system /tmp/ant_maze.xml). That leads to crashes if several processes try to instantiate an environment.

Code example

import gymnasium as gym

env = gym.vector.AsyncVectorEnv([lambda: gym.make("PointMaze_Open-v3") for _ in range(10)])

This produces

ValueError: XML parse error 13:
Error=XML_ERROR_EMPTY_DOCUMENT ErrorID=13 (0xd) Line number=0

among other parsing errors.

System Info

I am running Ubuntu 20.04.06
Gymnasium was installed via pip.
Python 3.8.10

Additional context
It looks like the problem is caused by the use of path.dirname(tmp_dir) here which effectively discards the temporary directory created in the line above.

Thank you for your help!

Checklist

I have checked that there is no similar issue in the repo (required)

[Bug Report] Links on the README.md are broken

Describe the bug

The links on the README.md page are broken.

[Bug Report] Setting goal_cell in point_maze causes an error

Describe the bug
Indexing error in point_maze assertion. When I pass the "goal_cell" as a 2D numpy array via the options dict as required here, I get this error:

  File "/home/vasan/RL/lib/python3.10/site-packages/gymnasium_robotics/envs/maze/maze.py", line 294, in reset
    self.maze.maze_map[options["goal_cell"][1], options["goal_cell"][0]]
TypeError: list indices must be integers or slices, not tuple

Solution
It's fairly simple. The array indexing code is broken. The assertion should be:

self.maze.maze_map[options["reset_cell"][1]][options["reset_cell"][0]] != 1

System Info

pip install within a conda env
MacOS
Python 3.10

Checklist

I have checked that there is no similar issue in the repo (required)

[Proposal] Add `compute_done` method on `GoalEnv`

Proposal

Since GoalEnv is used to use hindsight experience replay and most methods use done flag for value computation, it is relevant to add a compute_done method in the same spirit as compute_reward for better relabeling of experiences.

Motivation

It would enables to increase the learning results, see this comment DLR-RM/stable-baselines3#627 (comment)

Pitch

Add an abstract method in GoalEnv:

@abstractmethod
    def compute_done(self, achieved_goal, desired_goal, info):
        ...

Checklist

I have checked that there is no similar issue in the repo (required)

May I know how I can create a new fetechenv?

Hello there. I'm new to gym and RL. I would like to know how to create a new fetchenv? Since I saw that the code in gym is no longer working anymore, like there is no more gym.make('FetchReach-v0').

[Question] Which of the MAMuJoCo environments are even "solvable"?

Question

TL;DR: do you have baselines for performance on the environments using some popular MARL algorithm, say MADDPG or other?

Hi there, first of all, thanks for maintaining MAMuJoCo. I have been experimenting with it for a few weeks now but am struggling to "solve" several of the scenarios using MATD3 / MADDPG. I was wonder if you have any baselines for the environments, i.e., demonstrated that they can be "solved" using some MARL algorithm? By "solved" I just mean some non-trivial return. In particular, my algorithm quickly learn to get a score of around 800 and 1000 on Ant and HalfCheetah respectively but fail to break out out that local optima until I added qvel,qpos to the global_categories. After adding qvel,qpos I now get scores ~3000 and ~6000 respectively. I originally tried this because I suspected there was some important information missing in the agent observations after I reduced the problem to a single agent task on the joint-observation and joint-action and my TD3 implementation could not solve it.

I am now struggling to "solve" 2-agent Walker and 3-agent Hopper. I tried adding more values to the global_categories (qvel,qpos,cinert,cvel,qfrc_actuator,cfrc_ext) but my algorithm seems stuck around the ~500 and ~200 return mark. Because of my experience with Ant and HalfCheetah I fear there is some important information omitted from the joint-observation, making it impossible to solve.

To hopefully rule this out and narrow down the problem to a bug in my implementation I was hoping you had some kind of baseline for performance on these environments. I tried to refer to the results reported in other papers using MAMuJoCo but they all seem to use non-default settings for the scenarios which in some cases make the environment no longer a decentralised partially observed multi-agent environment. For example this paper gives each of the agents access to the state of the environment as their observation. I would like to avoid this and only give agents access to their partial local observations. However, I feel that if a single agent RL algorithm can't solve the tasks on the joint-observation, then its unrealistic to expect a MARL algorithm to succeed. What do you think?

I look forward to hearing from you.

Creation of Custom FetchObstaclePickAndPlace environment faces issue with adding obstacle data in the observation of the environment

Hello dear members of the Farama Team!

I have made a fork of the Gymnasium-Robotics API. I am working on creating a custom environment, based on the 'FetchPickAndPlace-v2' environment. This new environment will introduce an obstacle in the simulation. I have created the appropriate .xml file that introduces the obstacle, which is named 'objstacle0'. For this environment I wanted to add to the original observation of the 'FetchPickAnPlace' environment the following:

The position of the obstacle in the 3D space.
The relative position of the robot gripper and the obstacle.
The relative position of the object and the obstacle.

Here is the overwritten version of the 'generate_mujoco_observations()' method that I am using:

def generate_mujoco_observations(self):
# positions
grip_pos = self._utils.get_site_xpos(self.model, self.data, "robot0:grip")

    dt = self.n_substeps * self.model.opt.timestep
    grip_velp = (
        self._utils.get_site_xvelp(self.model, self.data, "robot0:grip") * dt
    )

    robot_qpos, robot_qvel = self._utils.robot_get_obs(
        self.model, self.data, self._model_names.joint_names
    )
    if self.has_object:
        object_pos = self._utils.get_site_xpos(self.model, self.data, "object0")
        # rotations
        object_rot = rotations.mat2euler(
            self._utils.get_site_xmat(self.model, self.data, "object0")
        )
        # velocities
        object_velp = (
            self._utils.get_site_xvelp(self.model, self.data, "object0") * dt
        )
        object_velr = (
            self._utils.get_site_xvelr(self.model, self.data, "object0") * dt
        )
        # gripper state
        object_rel_pos = object_pos - grip_pos
        object_velp -= grip_velp
    else:
        object_pos = (
            object_rot
        ) = object_velp = object_velr = object_rel_pos = np.zeros(0)
    gripper_state = robot_qpos[-2:]

    gripper_vel = (
        robot_qvel[-2:] * dt
    )  # change to a scalar if the gripper is made symmetric
    
    # Extract the positions of the gripper and the object
    #grip_pos = self._utils.get_site_xpos(self.model, self.data, "robot0:grip")
    #object_pos = self._utils.get_site_xpos(self.model, self.data, "object0")
    
    # Calculate the obstacle's position (assuming its site name is "obstacle")
    obstacle_pos = self._utils.get_site_xpos(self.model, self.data, "obstacle0")
    
    # Calculate the relative positions
    gripper_to_obstacle = obstacle_pos - grip_pos
    object_to_obstacle = obstacle_pos - object_pos
    
    # Extend the original observations with the relative positions
    #extended_observations = observations + (gripper_to_obstacle,) + (object_to_obstacle,)        
    return (
        grip_pos,
        object_pos,
        object_rel_pos,
        gripper_state,
        object_rot,
        object_velp,
        object_velr,
        grip_velp,
        gripper_vel,
        obstacle_pos,
        gripper_to_obstacle,
        object_to_obstacle
    )

And here the code of the obstacle_pick_and_place.xml:

<include file="shared.xml" />

<worldbody>
	<geom name="floor0" pos="0.8 0.75 0" size="0.85 0.7 1" type="plane" condim="3" material="floor_mat" />
	<body name="floor0" pos="0.8 0.75 0">
		<site name="target0" pos="0 0 0.5" size="0.02 0.02 0.02" rgba="1 0 0 1" type="sphere" />
	</body>

	<include file="robot.xml" />
	
	<body pos="1.3 0.75 0.2" name="table0">
		<geom size="0.25 0.35 0.2" type="box" mass="2000" material="table_mat" />
	</body>

	<body pos="1.3 0.75 0.48" name="obstacle0">
		<geom  size="0.025 0.35 0.075" type="box" rgba="0.6 0.3 0 1" />
	</body>
	
	<body name="object0" pos="0.025 0.025 0.025">
		<joint name="object0:joint" type="free" damping="0.01" />
		<geom size="0.025 0.025 0.025" type="box" condim="3" name="object0" material="block_mat" mass="2" />
		<site name="object0" pos="0 0 0" size="0.02 0.02 0.02" rgba="1 0 0 1" type="sphere" />
	</body>

	<light directional="true" ambient="0.2 0.2 0.2" diffuse="0.8 0.8 0.8" specular="0.3 0.3 0.3" castshadow="false" pos="0 0 4" dir="0 0 -1" name="light0" />
</worldbody>

<actuator>
	<position ctrllimited="true" ctrlrange="0 0.2" joint="robot0:l_gripper_finger_joint" kp="30000" name="robot0:l_gripper_finger_joint" user="1" />
	<position ctrllimited="true" ctrlrange="0 0.2" joint="robot0:r_gripper_finger_joint" kp="30000" name="robot0:r_gripper_finger_joint" user="1" />
</actuator>

Could you please help me debug and fix this behaviour ?

Thank you very much in advance for all the valuable help and support !!!

Kind regards,

Christos Peridis

	object_qpos = self._utils.get_joint_qpos(
	self.model, self.data, "object0:joint"
	)
	assert object_qpos.shape == (7,)
	object_qpos[:2] = object_xpos
	self._utils.set_joint_qpos(
	self.model, self.data, "object0:joint", object_qpos
	)

farama-foundation / gymnasium-robotics Goto Github PK

gymnasium-robotics's People

Contributors

Stargazers

Watchers

Forkers

gymnasium-robotics's Issues

Adapting Fetch Robot for the FrankaKitchen Task

Question

Proposal

Motivation

Pitch

Alternatives

Additional context

Checklist

Question

Proposal

Pitch

Checklist

1) AntMaze sparse reward always zero

2) AntMaze can reset into a terminal state

Checklist

Checklist

Checklist

Proposal

Git version

pip version

Checklist

Question

Question

Question

Checklist

Checklist

Proposal

Checklist

Proposal 0

Proposal 1

Proposal 2

Checklist

Proposal

Motivation

Pitch

Checklist

Question

Checklist

Solution?

Checklist

Checklist

Proposal

Motivation

Pitch

Checklist

Question

Recommend Projects

Recommend Topics

Recommend Org