mihdalal / raps Goto Github PK

[NeurIPS 2021] PyTorch Code for Accelerating Robotic Reinforcement Learning with Parameterized Action Primitives

Home Page: https://mihdalal.github.io/raps/

License: MIT License

Python 75.72% Dockerfile 0.10% Shell 0.32% Makefile 0.05% Jupyter Notebook 22.62% CMake 0.01% CSS 0.16% JavaScript 0.55% HTML 0.47%

deep-learning reinforcement-learning robotics primitives artificial-intelligence

raps's People

Stargazers

Watchers

Forkers

wx-b idate96 r-ceph zss1995 feup-jmc papercut-linkin yaoxt3 kejiachen yc-liang tarunc20 marinbao

raps's Issues

Erros when testing

Hello and thanks for your sharing of this amazing work!
I'm trying to run the code you recommended
python experiments/kitchen/dreamer/dreamer_v2_single_task_primitives.py --mode here_no_doodad --env microwave
however, i found an error which is not mentioned previously in other issues

2022-06-08 17:39:37.331497 CST | Variant:
2022-06-08 17:39:37.332199 CST | {
  "env_name": "microwave",
  "algorithm": "DreamerV2",
  "version": "normal",
  "replay_buffer_size": 500000,
  "algorithm_kwargs": {
    "num_epochs": 250,
    "num_eval_steps_per_epoch": 30,
    "min_num_steps_before_training": 2500,
    "num_pretrain_steps": 100,
    "max_path_length": 5,
    "batch_size": 417,
    "num_expl_steps_per_train_loop": 30,
    "num_train_loops_per_epoch": 40,
    "num_trains_per_train_loop": 10
  },
  "use_raw_actions": false,
  "env_suite": "kitchen",
  "pass_render_kwargs": true,
  "save_video": true,
  "env_kwargs": {
    "dense": false,
    "image_obs": true,
    "action_scale": 1.4,
    "use_workspace_limits": true,
    "control_mode": "primitives",
    "usage_kwargs": {
      "use_dm_backend": true,
      "use_raw_action_wrappers": false,
      "use_image_obs": true,
      "max_path_length": 5,
      "unflatten_images": false
    },
    "image_kwargs": {}
  },
  "actor_kwargs": {
    "discrete_continuous_dist": true,
    "init_std": 0.0,
    "num_layers": 4,
    "min_std": 0.1,
    "dist": "tanh_normal_dreamer_v1"
  },
  "vf_kwargs": {
    "num_layers": 3
  },
  "model_kwargs": {
    "model_hidden_size": 400,
    "stochastic_state_size": 50,
    "deterministic_state_size": 200,
    "embedding_size": 1024,
    "rssm_hidden_size": 200,
    "reward_num_layers": 2,
    "pred_discount_num_layers": 3,
    "gru_layer_norm": true,
    "std_act": "sigmoid2"
  },
  "trainer_kwargs": {
    "adam_eps": 1e-05,
    "discount": 0.8,
    "lam": 0.95,
    "forward_kl": false,
    "free_nats": 1.0,
    "pred_discount_loss_scale": 10.0,
    "kl_loss_scale": 0.0,
    "transition_loss_scale": 0.8,
    "actor_lr": 8e-05,
    "vf_lr": 8e-05,
    "world_model_lr": 0.0003,
    "reward_loss_scale": 2.0,
    "use_pred_discount": true,
    "policy_gradient_loss_scale": 1.0,
    "actor_entropy_loss_schedule": "1e-4",
    "target_update_period": 100,
    "detach_rewards": false,
    "imagination_horizon": 5
  },
  "num_expl_envs": 5,
  "num_eval_envs": 1,
  "expl_amount": 0.3,
  "seed": "23764",
  "exp_id": "0",
  "exp_prefix": "06-08-test",
  "instance_type": "None",
  "exp_name": "06-08-test_2022_06_08_17_39_37_0000--s-23764"
}
/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
Process SpawnProcess-1:
Traceback (most recent call last):
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/home/lwt/github_test/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "/home/lwt/github_test/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "/home/lwt/github_test/raps/rlkit/rlkit/envs/primitives_make_env.py", line 129, in make_env
    env = make_base_kitchen_env(env_name, env_kwargs_new)
  File "/home/lwt/github_test/raps/rlkit/rlkit/envs/primitives_make_env.py", line 99, in make_base_kitchen_env
    env = ALL_KITCHEN_ENVIRONMENTS[env_class](**env_kwargs)
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/kitchen_envs.py", line 36, in __init__
    super(KitchenBase, self).__init__(**kwargs)
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/franka/kitchen_multitask_v0.py", line 771, in __init__
    super(KitchenTaskRelaxV1, self).__init__(**kwargs)
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/franka/kitchen_multitask_v0.py", line 216, in __init__
    distance=2.2, lookat=[-0.2, 0.5, 2.0], azimuth=70, elevation=-35
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/robot_env.py", line 82, in __init__
    model_path, frame_skip, camera_settings=camera_settings
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/mujoco_env.py", line 67, in __init__
    camera_settings=camera_settings,
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 68, in __init__
    self.sim = dm_mujoco.Physics.from_xml_path(model_file)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/engine.py", line 443, in from_xml_path
    return cls.from_model(model)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/engine.py", line 407, in from_model
    return cls(data)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/engine.py", line 122, in __init__
    self._reload_from_data(data)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/engine.py", line 389, in _reload_from_data
    data=index.struct_indexer(self.data, 'mjdata', axis_indexers),)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/index.py", line 623, in struct_indexer
    attr = getattr(struct, field_name)
AttributeError: 'MjData' object has no attribute 'qacc_unc'
Process SpawnProcess-2:
Traceback (most recent call last):
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/home/lwt/github_test/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "/home/lwt/github_test/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "/home/lwt/github_test/raps/rlkit/rlkit/envs/primitives_make_env.py", line 129, in make_env
    env = make_base_kitchen_env(env_name, env_kwargs_new)
  File "/home/lwt/github_test/raps/rlkit/rlkit/envs/primitives_make_env.py", line 99, in make_base_kitchen_env
    env = ALL_KITCHEN_ENVIRONMENTS[env_class](**env_kwargs)
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/kitchen_envs.py", line 36, in __init__
    super(KitchenBase, self).__init__(**kwargs)
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/franka/kitchen_multitask_v0.py", line 771, in __init__
    super(KitchenTaskRelaxV1, self).__init__(**kwargs)
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/franka/kitchen_multitask_v0.py", line 216, in __init__
    distance=2.2, lookat=[-0.2, 0.5, 2.0], azimuth=70, elevation=-35
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/robot_env.py", line 82, in __init__
    model_path, frame_skip, camera_settings=camera_settings
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/mujoco_env.py", line 67, in __init__
    camera_settings=camera_settings,
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 68, in __init__
    self.sim = dm_mujoco.Physics.from_xml_path(model_file)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/engine.py", line 443, in from_xml_path
    return cls.from_model(model)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/engine.py", line 407, in from_model
    return cls(data)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/engine.py", line 122, in __init__
    self._reload_from_data(data)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/engine.py", line 389, in _reload_from_data
    data=index.struct_indexer(self.data, 'mjdata', axis_indexers),)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/index.py", line 623, in struct_indexer
    attr = getattr(struct, field_name)
AttributeError: 'MjData' object has no attribute 'qacc_unc'
Process SpawnProcess-5:
Traceback (most recent call last):
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/home/lwt/github_test/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "/home/lwt/github_test/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "/home/lwt/github_test/raps/rlkit/rlkit/envs/primitives_make_env.py", line 129, in make_env
    env = make_base_kitchen_env(env_name, env_kwargs_new)
  File "/home/lwt/github_test/raps/rlkit/rlkit/envs/primitives_make_env.py", line 99, in make_base_kitchen_env
    env = ALL_KITCHEN_ENVIRONMENTS[env_class](**env_kwargs)
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/kitchen_envs.py", line 36, in __init__
    super(KitchenBase, self).__init__(**kwargs)
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/franka/kitchen_multitask_v0.py", line 771, in __init__
    super(KitchenTaskRelaxV1, self).__init__(**kwargs)
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/franka/kitchen_multitask_v0.py", line 216, in __init__
    distance=2.2, lookat=[-0.2, 0.5, 2.0], azimuth=70, elevation=-35
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/robot_env.py", line 82, in __init__
    model_path, frame_skip, camera_settings=camera_settings
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/mujoco_env.py", line 67, in __init__
    camera_settings=camera_settings,
  File "/home/lwt/github_test/raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 68, in __init__
    self.sim = dm_mujoco.Physics.from_xml_path(model_file)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/engine.py", line 443, in from_xml_path
    return cls.from_model(model)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/engine.py", line 407, in from_model
    return cls(data)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/engine.py", line 122, in __init__
    self._reload_from_data(data)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/engine.py", line 389, in _reload_from_data
    data=index.struct_indexer(self.data, 'mjdata', axis_indexers),)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/site-packages/dm_control/mujoco/index.py", line 623, in struct_indexer
    attr = getattr(struct, field_name)
AttributeError: 'MjData' object has no attribute 'qacc_unc'
Traceback (most recent call last):
  File "experiments/kitchen/dreamer/dreamer_v2_single_task_primitives.py", line 139, in <module>
    exp_id=exp_id,
  File "/home/lwt/github_test/raps/rlkit/rlkit/launchers/launcher_util.py", line 590, in run_experiment
    return run_experiment_here(method_call, **run_experiment_kwargs)
  File "/home/lwt/github_test/raps/rlkit/rlkit/launchers/launcher_util.py", line 168, in run_experiment_here
    return experiment_function(variant)
  File "/home/lwt/github_test/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 45, in experiment
    expl_env = StableBaselinesVecEnv(env_fns=env_fns, start_method="fork")
  File "/home/lwt/github_test/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 134, in __init__
    observation_space, action_space = self.remotes[0].recv()
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/multiprocessing/connection.py", line 407, in _recv_bytes
    buf = self._recv(4)
  File "/home/lwt/anaconda3/envs/raps/lib/python3.7/multiprocessing/connection.py", line 379, in _recv
    chunk = read(handle, remaining)
ConnectionResetError: [Errno 104] Connection reset by peer

I searched google for answers to solve the MjData problem, but nothing was found.
I wonder if you can give me some advice, thanks for your selfless assistance!

Connection reset by peer

Good day,

After trying the suggested solutions in the other two issues on the same topic, when running

python experiments/robosuite/dreamer/dreamer_v2_single_task_primitives_door.py --mode here_no_doodad --exp_prefix test

I still get

2022-12-06 19:05:46.506113 CET | Variant:
2022-12-06 19:05:46.506515 CET | {
  "algorithm": "DreamerV2",
  "version": "normal",
  "replay_buffer_size": 500000,
  "algorithm_kwargs": {
    "num_epochs": 1000,
    "num_eval_steps_per_epoch": 30,
    "min_num_steps_before_training": 2500,
    "num_pretrain_steps": 100,
    "max_path_length": 5,
    "batch_size": 417,
    "num_expl_steps_per_train_loop": 60,
    "num_train_loops_per_epoch": 20,
    "num_trains_per_train_loop": 20
  },
  "env_name": "Door",
  "use_raw_actions": false,
  "env_suite": "robosuite",
  "env_kwargs": {
    "robots": "Panda",
    "has_renderer": false,
    "has_offscreen_renderer": false,
    "use_camera_obs": false,
    "camera_heights": 64,
    "camera_widths": 64,
    "controller_configs": {
      "type": "OSC_POSE",
      "input_max": 1,
      "input_min": -1,
      "output_max": [
        0.1,
        0.1,
        0.1,
        0.5,
        0.5,
        0.5
      ],
      "output_min": [
        -0.1,
        -0.1,
        -0.1,
        -0.5,
        -0.5,
        -0.5
      ],
      "kp": 150,
      "damping_ratio": 1,
      "impedance_mode": "fixed",
      "kp_limits": [
        0,
        300
      ],
      "damping_ratio_limits": [
        0,
        10
      ],
      "position_limits": null,
      "orientation_limits": null,
      "uncouple_pos_ori": true,
      "control_delta": true,
      "interpolation": null,
      "ramp_ratio": 0.2
    },
    "horizon": 5,
    "control_freq": 20,
    "reward_shaping": false,
    "reset_action_space_kwargs": {
      "control_mode": "primitives",
      "action_scale": 1,
      "max_path_length": 5,
      "workspace_low": [
        -0.3,
        -0.4,
        0.8
      ],
      "workspace_high": [
        0.0,
        0.2,
        1.3
      ],
      "go_to_pose_iterations": 40,
      "camera_settings": {
        "distance": 0.9532481338137215,
        "lookat": [
          -0.30494697896254724,
          -0.4394507345054956,
          0.9492024838769221
        ],
        "azimuth": -125.859375,
        "elevation": -29.062499923165888
      }
    },
    "usage_kwargs": {
      "use_dm_backend": true,
      "max_path_length": 5
    },
    "image_kwargs": {}
  },
  "actor_kwargs": {
    "discrete_continuous_dist": true,
    "init_std": 0.0,
    "num_layers": 4,
    "min_std": 0.1,
    "dist": "tanh_normal_dreamer_v1"
  },
  "vf_kwargs": {
    "num_layers": 3
  },
  "model_kwargs": {
    "model_hidden_size": 400,
    "stochastic_state_size": 50,
    "deterministic_state_size": 200,
    "embedding_size": 1024,
    "rssm_hidden_size": 200,
    "reward_num_layers": 2,
    "pred_discount_num_layers": 3,
    "gru_layer_norm": true,
    "std_act": "sigmoid2"
  },
  "trainer_kwargs": {
    "adam_eps": 1e-05,
    "discount": 0.8,
    "lam": 0.95,
    "forward_kl": false,
    "free_nats": 1.0,
    "pred_discount_loss_scale": 10.0,
    "kl_loss_scale": 0.0,
    "transition_loss_scale": 0.8,
    "actor_lr": 8e-05,
    "vf_lr": 8e-05,
    "world_model_lr": 0.0003,
    "reward_loss_scale": 2.0,
    "use_pred_discount": true,
    "policy_gradient_loss_scale": 1.0,
    "actor_entropy_loss_schedule": "1e-4",
    "target_update_period": 100,
    "detach_rewards": false,
    "imagination_horizon": 5
  },
  "num_expl_envs": 10,
  "num_eval_envs": 1,
  "expl_amount": 0.3,
  "pass_render_kwargs": true,
  "save_video": true,
  "seed": "31079",
  "exp_id": "0",
  "exp_prefix": "12-06-test",
  "instance_type": "None",
  "exp_name": "12-06-test_2022_12_06_19_05_46_0000--s-31079"
}
Process SpawnProcess-7:
Traceback (most recent call last):
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "/home/dve/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_make_env.py", line 104, in make_env
    from rlkit.envs.primitives_wrappers import (
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_wrappers.py", line 2, in <module>
    import mujoco_py
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/__init__.py", line 3, in <module>
    from mujoco_py.builder import cymj, ignore_mujoco_warnings, functions, MujocoException
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 510, in <module>
    cymj = load_cython_ext(mujoco_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 101, in load_cython_ext
    mod = load_dynamic_ext('cymj', cext_so_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 125, in load_dynamic_ext
    return loader.load_module()
  File "mujoco_py/cymj.pyx", line 1, in init mujoco_py.cymj
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
Process SpawnProcess-8:
Traceback (most recent call last):
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "/home/dve/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_make_env.py", line 104, in make_env
    from rlkit.envs.primitives_wrappers import (
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_wrappers.py", line 2, in <module>
    import mujoco_py
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/__init__.py", line 3, in <module>
    from mujoco_py.builder import cymj, ignore_mujoco_warnings, functions, MujocoException
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 510, in <module>
    cymj = load_cython_ext(mujoco_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 101, in load_cython_ext
    mod = load_dynamic_ext('cymj', cext_so_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 125, in load_dynamic_ext
    return loader.load_module()
  File "mujoco_py/cymj.pyx", line 1, in init mujoco_py.cymj
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
Process SpawnProcess-10:
Traceback (most recent call last):
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "/home/dve/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_make_env.py", line 104, in make_env
    from rlkit.envs.primitives_wrappers import (
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_wrappers.py", line 2, in <module>
    import mujoco_py
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/__init__.py", line 3, in <module>
    from mujoco_py.builder import cymj, ignore_mujoco_warnings, functions, MujocoException
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 510, in <module>
    cymj = load_cython_ext(mujoco_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 101, in load_cython_ext
    mod = load_dynamic_ext('cymj', cext_so_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 125, in load_dynamic_ext
    return loader.load_module()
  File "mujoco_py/cymj.pyx", line 1, in init mujoco_py.cymj
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
Process SpawnProcess-2:
Traceback (most recent call last):
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "/home/dve/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_make_env.py", line 104, in make_env
    from rlkit.envs.primitives_wrappers import (
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_wrappers.py", line 2, in <module>
    import mujoco_py
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/__init__.py", line 3, in <module>
    from mujoco_py.builder import cymj, ignore_mujoco_warnings, functions, MujocoException
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 510, in <module>
    cymj = load_cython_ext(mujoco_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 101, in load_cython_ext
    mod = load_dynamic_ext('cymj', cext_so_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 125, in load_dynamic_ext
    return loader.load_module()
  File "mujoco_py/cymj.pyx", line 1, in init mujoco_py.cymj
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
Process SpawnProcess-3:
Traceback (most recent call last):
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "/home/dve/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_make_env.py", line 104, in make_env
    from rlkit.envs.primitives_wrappers import (
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_wrappers.py", line 2, in <module>
    import mujoco_py
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/__init__.py", line 3, in <module>
    from mujoco_py.builder import cymj, ignore_mujoco_warnings, functions, MujocoException
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 510, in <module>
    cymj = load_cython_ext(mujoco_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 101, in load_cython_ext
    mod = load_dynamic_ext('cymj', cext_so_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 125, in load_dynamic_ext
    return loader.load_module()
  File "mujoco_py/cymj.pyx", line 1, in init mujoco_py.cymj
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
Process SpawnProcess-1:
Traceback (most recent call last):
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "/home/dve/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_make_env.py", line 104, in make_env
    from rlkit.envs.primitives_wrappers import (
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_wrappers.py", line 2, in <module>
    import mujoco_py
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/__init__.py", line 3, in <module>
    from mujoco_py.builder import cymj, ignore_mujoco_warnings, functions, MujocoException
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 510, in <module>
    cymj = load_cython_ext(mujoco_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 101, in load_cython_ext
    mod = load_dynamic_ext('cymj', cext_so_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 125, in load_dynamic_ext
    return loader.load_module()
  File "mujoco_py/cymj.pyx", line 1, in init mujoco_py.cymj
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
Process SpawnProcess-6:
Traceback (most recent call last):
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "/home/dve/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_make_env.py", line 104, in make_env
    from rlkit.envs.primitives_wrappers import (
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_wrappers.py", line 2, in <module>
    import mujoco_py
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/__init__.py", line 3, in <module>
    from mujoco_py.builder import cymj, ignore_mujoco_warnings, functions, MujocoException
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 510, in <module>
    cymj = load_cython_ext(mujoco_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 101, in load_cython_ext
    mod = load_dynamic_ext('cymj', cext_so_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 125, in load_dynamic_ext
    return loader.load_module()
  File "mujoco_py/cymj.pyx", line 1, in init mujoco_py.cymj
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
Process SpawnProcess-4:
Traceback (most recent call last):
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "/home/dve/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_make_env.py", line 104, in make_env
    from rlkit.envs.primitives_wrappers import (
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_wrappers.py", line 2, in <module>
    import mujoco_py
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/__init__.py", line 3, in <module>
    from mujoco_py.builder import cymj, ignore_mujoco_warnings, functions, MujocoException
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 510, in <module>
    cymj = load_cython_ext(mujoco_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 101, in load_cython_ext
    mod = load_dynamic_ext('cymj', cext_so_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 125, in load_dynamic_ext
    return loader.load_module()
  File "mujoco_py/cymj.pyx", line 1, in init mujoco_py.cymj
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
Process SpawnProcess-5:
Traceback (most recent call last):
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "/home/dve/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_make_env.py", line 104, in make_env
    from rlkit.envs.primitives_wrappers import (
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_wrappers.py", line 2, in <module>
    import mujoco_py
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/__init__.py", line 3, in <module>
    from mujoco_py.builder import cymj, ignore_mujoco_warnings, functions, MujocoException
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 510, in <module>
    cymj = load_cython_ext(mujoco_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 101, in load_cython_ext
    mod = load_dynamic_ext('cymj', cext_so_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 125, in load_dynamic_ext
    return loader.load_module()
  File "mujoco_py/cymj.pyx", line 1, in init mujoco_py.cymj
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
Process SpawnProcess-9:
Traceback (most recent call last):
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "/home/dve/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_make_env.py", line 104, in make_env
    from rlkit.envs.primitives_wrappers import (
  File "/home/dve/raps/rlkit/rlkit/envs/primitives_wrappers.py", line 2, in <module>
    import mujoco_py
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/__init__.py", line 3, in <module>
    from mujoco_py.builder import cymj, ignore_mujoco_warnings, functions, MujocoException
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 510, in <module>
    cymj = load_cython_ext(mujoco_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 101, in load_cython_ext
    mod = load_dynamic_ext('cymj', cext_so_path)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/site-packages/mujoco_py/builder.py", line 125, in load_dynamic_ext
    return loader.load_module()
  File "mujoco_py/cymj.pyx", line 1, in init mujoco_py.cymj
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
Traceback (most recent call last):
  File "/home/dve/raps/rlkit/experiments/robosuite/dreamer/dreamer_v2_single_task_primitives_door.py", line 167, in <module>
    run_experiment(
  File "/home/dve/raps/rlkit/rlkit/launchers/launcher_util.py", line 590, in run_experiment
    return run_experiment_here(method_call, **run_experiment_kwargs)
  File "/home/dve/raps/rlkit/rlkit/launchers/launcher_util.py", line 168, in run_experiment_here
    return experiment_function(variant)
  File "/home/dve/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 45, in experiment
    expl_env = StableBaselinesVecEnv(env_fns=env_fns, start_method="fork")
  File "/home/dve/raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 134, in __init__
    observation_space, action_space = self.remotes[0].recv()
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/connection.py", line 255, in recv
    buf = self._recv_bytes()
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/connection.py", line 419, in _recv_bytes
    buf = self._recv(4)
  File "/home/dve/anaconda3/envs/pt1.7/lib/python3.9/multiprocessing/connection.py", line 384, in _recv
    chunk = read(handle, remaining)
ConnectionResetError: [Errno 104] Connection reset by peer

Note that my installed packages are:

Package                  Version       
------------------------ ----------------- 
a2c-ppo-acktr            0.0.1     
absl-py                  1.3.0
awscli                   1.27.23
boto                     2.49.0
boto3                    1.26.23
botocore                 1.29.23
certifi                  2022.9.24
cffi                     1.15.1
charset-normalizer       2.1.1
click                    8.0.1
cloudpickle              1.6.0
colorama                 0.4.4
contourpy                1.0.6
cycler                   0.11.0
Cython                   0.29.32
d4rl                     1.1               
dm-control               1.0.0
dm-env                   1.5
dm-tree                  0.1.7
docutils                 0.16
doodad                   0.2.1.dev0        /home/dve/raps/doodad
fasteners                0.18
Flask                    2.1.2
fonttools                4.38.0
future                   0.18.2
glfw                     2.5.5
gtimer                   1.0.0b5
gym                      0.18.3
gym-notices              0.0.8
h5py                     3.7.0
idna                     3.4
imageio                  2.9.0
importlib-metadata       4.13.0
itsdangerous             2.1.2
Jinja2                   3.1.2
jmespath                 1.0.1
kiwisolver               1.4.4
labmaze                  1.0.6
llvmlite                 0.36.0
lxml                     4.9.1
MarkupSafe               2.1.1
matplotlib               3.3.4
metaworld                0.0.0    
mjrl                     1.0.0
mujoco-py                2.0.2.13
networkx                 2.8.8
numba                    0.53.1
numpy                    1.21.6
numpy-quaternion         2021.4.5.14.42.35
nvidia-cublas-cu11       11.10.3.66
nvidia-cuda-nvrtc-cu11   11.7.99
nvidia-cuda-runtime-cu11 11.7.99
nvidia-cudnn-cu11        8.5.0.96
opencv-python            4.5.2.52
packaging                21.3
pandas                   1.5.2
Pillow                   8.2.0
pip                      22.2.2
plotly                   4.0.0
protobuf                 4.21.10
pyasn1                   0.4.8
pybullet                 3.2.5
pycparser                2.21
pyglet                   1.5.15
PyOpenGL                 3.1.6
pyparsing                2.4.7
python-dateutil          2.8.1
pytz                     2022.6
PyWavelets               1.4.1
PyYAML                   5.4.1
rad                      1.0.0          
requests                 2.28.1
retrying                 1.3.4
rlkit                    0.2.1.dev0   
robosuite                1.2.0          
rsa                      4.7.2
s3transfer               0.6.0
scikit-image             0.17.2
scipy                    1.9.3
setuptools               65.5.0
six                      1.16.0
stable-baselines3        1.0
termcolor                1.1.0
tifffile                 2022.10.10
torch                    1.7.1
torchvision              0.8.2
tqdm                     4.64.1
typing_extensions        4.4.0
urllib3                  1.26.13
viskit                   0.1     
Werkzeug                 2.2.2
wheel                    0.37.1
zipp                     3.11.0

Any help would be much appreciated!
Kind regards

RuntimeError: nvrtc: error: failed to load builtins for compute_80

Hi! @mihdalal! Thanks again for checking this issue.

When I tried to train the robosuite lift task, I met the following error:

/home/xzc/miniforge3/envs/raps/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
/home/xzc/miniforge3/envs/raps/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
/home/xzc/miniforge3/envs/raps/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
/home/xzc/miniforge3/envs/raps/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
/home/xzc/miniforge3/envs/raps/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
/home/xzc/miniforge3/envs/raps/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
/home/xzc/miniforge3/envs/raps/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
/home/xzc/miniforge3/envs/raps/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
/home/xzc/miniforge3/envs/raps/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
/home/xzc/miniforge3/envs/raps/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
/home/xzc/miniforge3/envs/raps/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
TRAINING
Traceback (most recent call last):
  File "experiments/robosuite/dreamer/dreamer_v2_single_task_primitives_lift.py", line 182, in <module>
    exp_id=exp_id,
  File "/home/xzc/Documents/raps/rlkit/rlkit/launchers/launcher_util.py", line 590, in run_experiment
    return run_experiment_here(method_call, **run_experiment_kwargs)
  File "/home/xzc/Documents/raps/rlkit/rlkit/launchers/launcher_util.py", line 168, in run_experiment_here
    return experiment_function(variant)
  File "/home/xzc/Documents/raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 196, in experiment
    algorithm.train()
  File "/home/xzc/Documents/raps/rlkit/rlkit/torch/model_based/rl_algorithm.py", line 52, in train
    self._train()
  File "/home/xzc/Documents/raps/rlkit/rlkit/torch/model_based/rl_algorithm.py", line 277, in _train
    self.num_eval_steps_per_epoch,
  File "/home/xzc/Documents/raps/rlkit/rlkit/torch/model_based/dreamer/path_collector.py", line 54, in collect_new_paths
    render_kwargs=self._render_kwargs,
  File "/home/xzc/Documents/raps/rlkit/rlkit/torch/model_based/dreamer/rollout_functions.py", line 63, in vec_rollout
    a, agent_info = agent.get_action(o_for_agent, **get_action_kwargs)
  File "/home/xzc/miniforge3/envs/raps/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 26, in decorate_context
    return func(*args, **kwargs)
  File "/home/xzc/Documents/raps/rlkit/rlkit/torch/model_based/dreamer/dreamer_policy.py", line 49, in get_action
    embed = self.world_model.encode(observation)
RuntimeError: nvrtc: error: failed to load builtins for compute_80.
nvrtc compilation failed: 

#define NAN __int_as_float(0x7fffffff)
#define POS_INFINITY __int_as_float(0x7f800000)
#define NEG_INFINITY __int_as_float(0xff800000)


template<typename T>
__device__ T maximum(T a, T b) {
  return isnan(a) ? a : (a > b ? a : b);
}

template<typename T>
__device__ T minimum(T a, T b) {
  return isnan(a) ? a : (a < b ? a : b);
}

extern "C" __global__
void func_1(float* t0, float* aten_sub_flat) {
{
  float v = __ldg(t0 + 512 * blockIdx.x + threadIdx.x);
  aten_sub_flat[512 * blockIdx.x + threadIdx.x] = v / 255.f - 0.5f;
}
}

It seems that this error is related to the cuda version?

I installed torch1.7.1+cu110 according to the requirements.txt with py3.7 on ubuntu 20.04. Besides, my cuda version is 11.2, and GPU (3070Ti) can be found with this torch version.

Is there a mismatch with my settings?

Kind thanks.

Errors when attempting to train

Hello and once again thanks for the amazing work!

Context

So far, I've managed to install the repo but running tests has proven difficult.

An issue I found early on was that running a sample training command like mentioned in the README didn't work as the program seemed to be unable to remember the environment passed by --env. This issue is mentioned in #1, and for the sake of consistency I will use the same command moving forward in these issues:
$ python experiments/kitchen/dreamer/dreamer_v2_single_task_primitives.py --mode here_no_doodad --exp_prefix <> --env microwave

With a bit of debugging I think I've managed to solve it, although this issue is not limited to dreamer_v2_single_task_primitives.py :

raps/rlkit/experiments/kitchen/dreamer/dreamer_v2_single_task_primitives.py

Line 117 in 23c7d95

search_space = {"env_name": [args.env]}

should actually be search_space = {"env_name": [args.exp_prefix]}.

Using sed for this replacement shows the following to be all the affected files (I can make a PR later if it seems like this issue is not exclusive to me):

experiments/kitchen/dreamer/dreamer_v2_multi_task_primitives.py
experiments/kitchen/dreamer/dreamer_v2_raw_actions.py
experiments/kitchen/dreamer/dreamer_v2_single_task_primitives.py
experiments/kitchen/dreamer/dreamer_v2_vices.py
experiments/kitchen/ppo/ppo_raw_actions.py
experiments/kitchen/ppo/ppo_vices.py
experiments/metaworld/dreamer/dreamer_v2_single_task_primitives.py
experiments/metaworld/dreamer/dreamer_v2_single_task_raw_actions.py
experiments/metaworld/dreamer/dreamer_v2_single_task_vices.py

Issue

With this problem out of the way, however, here is the real focus of this issue - when running the command again after solving the aforementioned problem, this is the output error (with similar results for other kitchen training scripts):

$ python experiments/kitchen/dreamer/dreamer_v2_single_task_primitives.py --mode here_no_doodad --exp_prefix <> --env microwave                            
duplicate key ignored: env_name
2022-05-14 14:51:12.216333 WEST | Variant:
2022-05-14 14:51:12.216544 WEST | {
  "env_name": "microwave",
  "algorithm": "DreamerV2",
  "version": "normal",
  "replay_buffer_size": 500000,
  "algorithm_kwargs": {
    "num_epochs": 250,
    "num_eval_steps_per_epoch": 30,
    "min_num_steps_before_training": 2500,
    "num_pretrain_steps": 100,
    "max_path_length": 5,
    "batch_size": 417,
    "num_expl_steps_per_train_loop": 30,
    "num_train_loops_per_epoch": 40,
    "num_trains_per_train_loop": 10
  },
  "use_raw_actions": false,
  "env_suite": "kitchen",
  "pass_render_kwargs": true,
  "save_video": true,
  "env_kwargs": {
    "dense": false,
    "image_obs": true,
    "action_scale": 1.4,
    "use_workspace_limits": true,
    "control_mode": "primitives",
    "usage_kwargs": {
      "use_dm_backend": true,
      "use_raw_action_wrappers": false,
      "use_image_obs": true,
      "max_path_length": 5,
      "unflatten_images": false
    },
    "image_kwargs": {}
  },
  "actor_kwargs": {
    "discrete_continuous_dist": true,
    "init_std": 0.0,
    "num_layers": 4,
    "min_std": 0.1,
    "dist": "tanh_normal_dreamer_v1"
  },
  "vf_kwargs": {
    "num_layers": 3
  },
  "model_kwargs": {
    "model_hidden_size": 400,
    "stochastic_state_size": 50,
    "deterministic_state_size": 200,
    "embedding_size": 1024,
    "rssm_hidden_size": 200,
    "reward_num_layers": 2,
    "pred_discount_num_layers": 3,
    "gru_layer_norm": true,
    "std_act": "sigmoid2"
  },
  "trainer_kwargs": {
    "adam_eps": 1e-05,
    "discount": 0.8,
    "lam": 0.95,
    "forward_kl": false,
    "free_nats": 1.0,
    "pred_discount_loss_scale": 10.0,
    "kl_loss_scale": 0.0,
    "transition_loss_scale": 0.8,
    "actor_lr": 8e-05,
    "vf_lr": 8e-05,
    "world_model_lr": 0.0003,
    "reward_loss_scale": 2.0,
    "use_pred_discount": true,
    "policy_gradient_loss_scale": 1.0,
    "actor_entropy_loss_schedule": "1e-4",
    "target_update_period": 100,
    "detach_rewards": false,
    "imagination_horizon": 5
  },
  "num_expl_envs": 5,
  "num_eval_envs": 1,
  "expl_amount": 0.3,
  "seed": "94249",
  "exp_id": "0",
  "exp_prefix": "05-14-microwave",
  "instance_type": "None",
  "exp_name": "05-14-microwave_2022_05_14_14_51_12_0000--s-94249"
}
~/anaconda3/envs/IKEA_2/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
~/anaconda3/envs/IKEA_2/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
~/anaconda3/envs/IKEA_2/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
~/anaconda3/envs/IKEA_2/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
Process SpawnProcess-1:
Traceback (most recent call last):
  File "~/anaconda3/envs/IKEA_2/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "~/anaconda3/envs/IKEA_2/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "~/xstar-raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "~/xstar-raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "~/xstar-raps/rlkit/rlkit/envs/primitives_make_env.py", line 129, in make_env
    env = make_base_kitchen_env(env_name, env_kwargs_new)
  File "~/xstar-raps/rlkit/rlkit/envs/primitives_make_env.py", line 99, in make_base_kitchen_env
    env = ALL_KITCHEN_ENVIRONMENTS[env_class](**env_kwargs)
  File "~/xstar-raps/d4rl/d4rl/kitchen/kitchen_envs.py", line 36, in __init__
    super(KitchenBase, self).__init__(**kwargs)
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/franka/kitchen_multitask_v0.py", line 771, in __init__
    super(KitchenTaskRelaxV1, self).__init__(**kwargs)
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/franka/kitchen_multitask_v0.py", line 216, in __init__
    distance=2.2, lookat=[-0.2, 0.5, 2.0], azimuth=70, elevation=-35
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/robot_env.py", line 82, in __init__
    model_path, frame_skip, camera_settings=camera_settings
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/mujoco_env.py", line 67, in __init__
    camera_settings=camera_settings,
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 70, in __init__
    self._patch_mjlib_accessors(self.model, self.sim.data)
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 303, in _patch_mjlib_accessors
    data.site_xvelp = site_xvelp()
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 286, in site_xvelp
    jacp = site_jacp().reshape((model.nsite, 3, model.nv))
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 282, in site_jacp
    mjlib.mj_jacSite(model.ptr, data.ptr, jacp_view, None, i)
TypeError: jacp should be of shape (3, nv)
~/anaconda3/envs/IKEA_2/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
Process SpawnProcess-2:
Traceback (most recent call last):
  File "~/anaconda3/envs/IKEA_2/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "~/anaconda3/envs/IKEA_2/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "~/xstar-raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "~/xstar-raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "~/xstar-raps/rlkit/rlkit/envs/primitives_make_env.py", line 129, in make_env
    env = make_base_kitchen_env(env_name, env_kwargs_new)
  File "~/xstar-raps/rlkit/rlkit/envs/primitives_make_env.py", line 99, in make_base_kitchen_env
    env = ALL_KITCHEN_ENVIRONMENTS[env_class](**env_kwargs)
  File "~/xstar-raps/d4rl/d4rl/kitchen/kitchen_envs.py", line 36, in __init__
    super(KitchenBase, self).__init__(**kwargs)
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/franka/kitchen_multitask_v0.py", line 771, in __init__
    super(KitchenTaskRelaxV1, self).__init__(**kwargs)
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/franka/kitchen_multitask_v0.py", line 216, in __init__
    distance=2.2, lookat=[-0.2, 0.5, 2.0], azimuth=70, elevation=-35
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/robot_env.py", line 82, in __init__
    model_path, frame_skip, camera_settings=camera_settings
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/mujoco_env.py", line 67, in __init__
    camera_settings=camera_settings,
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 70, in __init__
    self._patch_mjlib_accessors(self.model, self.sim.data)
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 303, in _patch_mjlib_accessors
    data.site_xvelp = site_xvelp()
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 286, in site_xvelp
    jacp = site_jacp().reshape((model.nsite, 3, model.nv))
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 282, in site_jacp
    mjlib.mj_jacSite(model.ptr, data.ptr, jacp_view, None, i)
TypeError: jacp should be of shape (3, nv)
Process SpawnProcess-4:
Traceback (most recent call last):
  File "~/anaconda3/envs/IKEA_2/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "~/anaconda3/envs/IKEA_2/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "~/xstar-raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 72, in _worker
    env = env_fn_wrapper.var()
  File "~/xstar-raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 42, in <lambda>
    lambda: primitives_make_env.make_env(env_suite, env_name, env_kwargs)
  File "~/xstar-raps/rlkit/rlkit/envs/primitives_make_env.py", line 129, in make_env
    env = make_base_kitchen_env(env_name, env_kwargs_new)
  File "~/xstar-raps/rlkit/rlkit/envs/primitives_make_env.py", line 99, in make_base_kitchen_env
    env = ALL_KITCHEN_ENVIRONMENTS[env_class](**env_kwargs)
  File "~/xstar-raps/d4rl/d4rl/kitchen/kitchen_envs.py", line 36, in __init__
    super(KitchenBase, self).__init__(**kwargs)
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/franka/kitchen_multitask_v0.py", line 771, in __init__
    super(KitchenTaskRelaxV1, self).__init__(**kwargs)
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/franka/kitchen_multitask_v0.py", line 216, in __init__
    distance=2.2, lookat=[-0.2, 0.5, 2.0], azimuth=70, elevation=-35
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/robot_env.py", line 82, in __init__
    model_path, frame_skip, camera_settings=camera_settings
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/mujoco_env.py", line 67, in __init__
    camera_settings=camera_settings,
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 70, in __init__
    self._patch_mjlib_accessors(self.model, self.sim.data)
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 303, in _patch_mjlib_accessors
    data.site_xvelp = site_xvelp()
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 286, in site_xvelp
    jacp = site_jacp().reshape((model.nsite, 3, model.nv))
  File "~/xstar-raps/d4rl/d4rl/kitchen/adept_envs/simulation/sim_robot.py", line 282, in site_jacp
    mjlib.mj_jacSite(model.ptr, data.ptr, jacp_view, None, i)
TypeError: jacp should be of shape (3, nv)
Traceback (most recent call last):
  File "experiments/kitchen/dreamer/dreamer_v2_single_task_primitives.py", line 139, in <module>
    exp_id=exp_id,
  File "~/xstar-raps/rlkit/rlkit/launchers/launcher_util.py", line 590, in run_experiment
    return run_experiment_here(method_call, **run_experiment_kwargs)
  File "~/xstar-raps/rlkit/rlkit/launchers/launcher_util.py", line 168, in run_experiment_here
    return experiment_function(variant)
  File "~/xstar-raps/rlkit/rlkit/torch/model_based/dreamer/experiments/kitchen_dreamer.py", line 45, in experiment
    expl_env = StableBaselinesVecEnv(env_fns=env_fns, start_method="fork")
  File "~/xstar-raps/rlkit/rlkit/envs/mujoco_vec_wrappers.py", line 134, in __init__
    observation_space, action_space = self.remotes[0].recv()
  File "~/anaconda3/envs/IKEA_2/lib/python3.7/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "~/anaconda3/envs/IKEA_2/lib/python3.7/multiprocessing/connection.py", line 407, in _recv_bytes
    buf = self._recv(4)
  File "~/anaconda3/envs/IKEA_2/lib/python3.7/multiprocessing/connection.py", line 379, in _recv
    chunk = read(handle, remaining)
ConnectionResetError: [Errno 104] Connection reset by peer

As I am not too well acquainted with the internals of MuJoCo, this would be very difficult for me to debug. I would very thankful if you could provide some insight in how to solve this error.

Contradictory dependencies make building impossible

Hello, and thanks for the cool repo on the paper.
I would like to use RAPS in another project but due to the contradictory dependencies in

raps/viskit/setup.py

Line 7 in ec6d54a

 REQUIRED_PACKAGES = ['Flask==1.0.2', 'matplotlib==2.0.2', 'plotly==4.0.0', 'numpy==1.16.0'] 

and

raps/requirements.txt

Line 4 in ec6d54a

mujoco_py==2.0.2.13

raps/requirements.txt

Line 12 in ec6d54a

matplotlib==3.3.4

raps/requirements.txt

Line 21 in ec6d54a

tensorboard==2.5.0

with the rest of the repo, I get the following errors when setting it up:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
tensorflow-gpu 2.8.0 requires numpy>=1.20, but you have numpy 1.16.0 which is incompatible.
tensorflow-gpu 2.8.0 requires tensorboard<2.9,>=2.8, but you have tensorboard 2.5.0 which is incompatible.
pywavelets 1.3.0 requires numpy>=1.17.3, but you have numpy 1.16.0 which is incompatible.
pandas 1.3.5 requires numpy>=1.17.3; platform_machine != "aarch64" and platform_machine != "arm64" and python_version < "3.10", but you have numpy 1.16.0 which is incompatible.
metaworld 0.0.0 requires numpy>=1.18, but you have numpy 1.16.0 which is incompatible.

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
tensorflow-gpu 2.8.0 requires tensorboard<2.9,>=2.8, but you have tensorboard 2.5.0 which is incompatible.
robosuite 1.2.0 requires mujoco-py==2.0.2.9, but you have mujoco-py 2.0.2.13 which is incompatible.
viskit 0.1 requires matplotlib==2.0.2, but you have matplotlib 3.3.4 which is incompatible.
viskit 0.1 requires numpy==1.16.0, but you have numpy 1.21.6 which is incompatible.

What should I do? Just ignore the version specification on the conflicting dependencies?

Aggressive swapping in robosuite training

Hello, and once again thanks for the great work on this repo!

This is not so much an issue with RAPS errors but rather something I've observed when training RAPS.
Unlike the other suites included with RAPS, robosuite has very high disk usage during training.
I am unsure whether this is an isolated issue, but there is a constant writing of data to swap of between 0 and 400 MB/s (with some peaks as high as 1200MB/s). This can be quite damaging to an SSD in the long-term (like in my setup), given the limited number of writes characteristic of these drives.

Is there something that can be done about this?

Hardware and time used for training?

Hi~
Could you share which hardware setup you used for training and roughly how long one training run took on that setup?
Thanks a lot !

mihdalal / raps Goto Github PK

raps's People

Stargazers

Watchers

Forkers

raps's Issues

Erros when testing

Connection reset by peer

RuntimeError: nvrtc: error: failed to load builtins for compute_80

Errors when attempting to train

Context

Issue

Contradictory dependencies make building impossible

Aggressive swapping in robosuite training

Hardware and time used for training?

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent