jiayuan-gu / hab-mobile-manipulation Goto Github PK

Hi @Jiayuan-Gu, thanks for sharing the nice implementation.
I notice that we can run evaluations on 64 training scenes. (correct me if I'm wrong)
Can you enlighten me on how to modify the config files to run evaluation on 20 test scenes and the macro variation scene?
Do I need to self-generate the episodes?

When I run this command, the terminal prompts me：

usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help

error: option --bullet not recognized

Thanks to the people who will be able to inform me of the solution in the future.🙏🙏🙏

We want to customize the objects in the environment and their placement positions and angles, similar to tidy_house_220417_train.json.gz file, but most tutorials explain how to import existing datasets and perform simulation rendering. Can you provide some guidance? Thank you very much.

I installed the project following your instructions.

I was trying to run the interactive play script, and I was expecting an environment to pop up in which I would be able to manipulate a robot using manual keys.

Instead the script runs, logs indicate that it runs successfully, but no interface like above comes up.

Am I using the script correctly?

Attaching the output log from running the script
play_log.txt

Hi, there is an issue when I try to generate the physical config.
ModuleNotFoundError: No module named 'habitat_sim._ext'
Is there any other ways to generate the config, or how can I fix this error?
Thank you for your generous help!

When I use the command about Evaluate a HAB (Home Assistant Benchmark) task from the README.md, all the commands show the same error:

[22:54:57:295573]:[Nav] PathFinder.cpp(386)::build : Building navmesh with 145 x 259 cells
[22:54:57:307122]:[Nav] PathFinder.cpp(656)::build : Created navmesh with 86 vertices 40 polygons
[22:54:57:307135]:[Sim] Simulator.cpp(920)::recomputeNavMesh : reconstruct navmesh successful
Skill begin.
Skill begin.
Traceback (most recent call last):
File "mobile_manipulation/eval_composite.py", line 328, in
main()
File "mobile_manipulation/eval_composite.py", line 220, in main
step_action = policy.act(ob)
File "/home/ubuntu/habitat/hab-mobile-manipulation/mobile_manipulation/methods/skill.py", line 127, in act
action = self.current_skill.act(obs, **kwargs)
File "/home/ubuntu/habitat/hab-mobile-manipulation/mobile_manipulation/methods/skill.py", line 127, in act
action = self.current_skill.act(obs, **kwargs)
File "/home/ubuntu/habitat/hab-mobile-manipulation/mobile_manipulation/methods/skills/rl_skills.py", line 67, in act
outputs = self.actor_critic.act(step_batch, deterministic=True)
File "/home/ubuntu/habitat/hab-mobile-manipulation/mobile_manipulation/ppo/policy.py", line 130, in act
net_outputs: Dict[str, torch.Tensor] = self.net(batch)
File "/home/ubuntu/anaconda3/envs/hab-mm/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "/home/ubuntu/habitat/hab-mobile-manipulation/mobile_manipulation/ppo/policies/cnn_policy.py", line 211, in forward
perception_embed = self.visual_encoder(cnn_input)
File "/home/ubuntu/anaconda3/envs/hab-mm/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "/home/ubuntu/habitat/hab-mobile-manipulation/mobile_manipulation/ppo/policies/cnn_policy.py", line 46, in forward
x = m(x)
File "/home/ubuntu/anaconda3/envs/hab-mm/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "/home/ubuntu/anaconda3/envs/hab-mm/lib/python3.7/site-packages/torch/nn/modules/linear.py", line 87, in forward
return F.linear(input, self.weight, self.bias)
File "/home/ubuntu/anaconda3/envs/hab-mm/lib/python3.7/site-packages/torch/nn/functional.py", line 1610, in linear
ret = torch.addmm(bias, input, weight.t())
RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling cublasSgemm( handle, opa, opb, m, n, k, &alpha, a, lda, b, ldb, &beta, c, ldc)

My device is RTX3090 and driver version is 535.154.05 and torch version : 1.5.1 and installed cuda version : 10.2 and CUDA Compute Capability: 8.6

Hi @Jiayuan-Gu. Thanks for sharing such a great project implementation!
I notice that in your paper you utilize Monolithic RL as the worst baseline comparison, may I ask how I can reproduce this baseline performance using your code (or how I can modify the contents of the config file)?

• hi, I create my conda env as required in instructions, which is python=3.7. when I tried to install habitat-sim, I got several issues. One of them is that when running python setup.py install --bullet --headless , there is issue said :

CMake Error: Error: generator : Ninja
Does not match the generator used previously: Unix Makefiles
Either remove the CMakeCache.txt file and CMakeFiles directory or choose a different binary directory.
Traceback (most recent call last):
File "setup.py", line 472, in
include_package_data=True,
File "/home/yvhongfei/anaconda3/envs/hab/lib/python3.7/site-packages/setuptools/init.py", line 87, in setup
return distutils.core.setup(**attrs)
File "/home/yvhongfei/anaconda3/envs/hab/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 185, in setup
return run_commands(dist)
File "/home/yvhongfei/anaconda3/envs/hab/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
dist.run_commands()
File "/home/yvhongfei/anaconda3/envs/hab/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
self.run_command(cmd)
File "/home/yvhongfei/anaconda3/envs/hab/lib/python3.7/site-packages/setuptools/dist.py", line 1208, in run_command
super().run_command(command)
File "/home/yvhongfei/anaconda3/envs/hab/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/home/yvhongfei/anaconda3/envs/hab/lib/python3.7/site-packages/setuptools/command/install.py", line 74, in run
self.do_egg_install()
File "/home/yvhongfei/anaconda3/envs/hab/lib/python3.7/site-packages/setuptools/command/install.py", line 123, in do_egg_install
self.run_command('bdist_egg')
File "/home/yvhongfei/anaconda3/envs/hab/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/home/yvhongfei/anaconda3/envs/hab/lib/python3.7/site-packages/setuptools/dist.py", line 1208, in run_command
super().run_command(command)
File "/home/yvhongfei/anaconda3/envs/hab/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/home/yvhongfei/anaconda3/envs/hab/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 165, in run
cmd = self.call_command('install_lib', warn_dir=0)
File "/home/yvhongfei/anaconda3/envs/hab/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 151, in call_command
self.run_command(cmdname)
File "/home/yvhongfei/anaconda3/envs/hab/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/home/yvhongfei/anaconda3/envs/hab/lib/python3.7/site-packages/setuptools/dist.py", line 1208, in run_command
super().run_command(command)
File "/home/yvhongfei/anaconda3/envs/hab/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/home/yvhongfei/anaconda3/envs/hab/lib/python3.7/site-packages/setuptools/command/install_lib.py", line 11, in run
self.build()
File "/home/yvhongfei/anaconda3/envs/hab/lib/python3.7/site-packages/setuptools/_distutils/command/install_lib.py", line 112, in build
self.run_command('build_ext')
File "/home/yvhongfei/anaconda3/envs/hab/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/home/yvhongfei/anaconda3/envs/hab/lib/python3.7/site-packages/setuptools/dist.py", line 1208, in run_command
super().run_command(command)
File "/home/yvhongfei/anaconda3/envs/hab/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "setup.py", line 262, in run
self.build_extension(ext)
File "setup.py", line 365, in build_extension
cwd=self.build_temp,
File "/home/yvhongfei/anaconda3/envs/hab/lib/python3.7/subprocess.py", line 363, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '-DBUILD_PYTHON_BINDINGS=ON', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/yvhongfei/Try/hab-mobile-manipulation/habitat-sim/build/lib.linux-x86_64-cpython-37/habitat_sim/_ext', '-DPYTHON_EXECUTABLE=/home/yvhongfei/anaconda3/envs/hab/bin/python', '-DCMAKE_EXPORT_COMPILE_COMMANDS=ON', '-DREL_BUILD_RPATH=ON', '-DCMAKE_BUILD_TYPE=RelWithDebInfo', '-GNinja', '-DBUILD_GUI_VIEWERS=OFF', '-DTARGET_HEADLESS=ON', '-DBUILD_TEST=OFF', '-DBUILD_WITH_BULLET=ON', '-DBUILD_WITH_VHACD=OFF', '-DBUILD_DATATOOL=OFF', '-DBUILD_WITH_CUDA=OFF', '-DBUILD_BASIS_COMPRESSOR=OFF', '/home/yvhongfei/Try/hab-mobile-manipulation/habitat-sim/src']' returned non-zero exit status 1.

what should I do?

Hi, I'm using your dokcer image to create a container and steup the habitat-lab and hab-mobile-manipulation. And everything is ready, including data and episodes. However, there comes an issue about the URDF file when I tried to run a play-demo.
(base) ubuntu@949561b4793d:~/work/hab-mobile-manipulation$ python habitat_extensions/tasks/rearrange/play.py
pybullet build time: Sep 22 2020 00:55:20
Neither ifconfig (ifconfig -a) nor ip (ip address show) commands are available, listing network interfaces is likely to fail
Loaded /home/ubuntu/work/hab-mobile-manipulation/configs/rearrange/tasks/play.yaml
Merging /home/ubuntu/work/hab-mobile-manipulation/configs/rearrange/tasks/base.yaml into /home/ubuntu/work/hab-mobile-manipulation/configs/rearrange/tasks/play.yaml
Loaded /home/ubuntu/work/hab-mobile-manipulation/configs/rearrange/tasks/base.yaml
Merging /home/ubuntu/work/hab-mobile-manipulation/configs/rearrange/tasks/__base__.py into /home/ubuntu/work/hab-mobile-manipulation/configs/rearrange/tasks/base.yaml
Loaded /home/ubuntu/work/hab-mobile-manipulation/configs/rearrange/tasks/__base__.py
2023-11-09 11:16:10,090 Initializing dataset RearrangeDataset-v0
2023-11-09 11:16:12,117 initializing sim RearrangeSim-v0
b3Warning[examples/Importers/ImportURDFDemo/UrdfFindMeshFile.h,102]:
habitat_extensions/assets/robots/hab_fetch/robots/hab_fetch_arm.urdf:12: cannot find '../meshes/base_link.dae' in any directory in urdf path
b3Error[examples/Importers/ImportURDFDemo/BulletUrdfImporter.cpp,121]:
Could not parse visual element for Link:b3Error[examples/Importers/ImportURDFDemo/BulletUrdfImporter.cpp,121]:
base_linkb3Error[examples/Importers/ImportURDFDemo/BulletUrdfImporter.cpp,121]:
failed to parse linkTraceback (most recent call last):
File "habitat_extensions/tasks/rearrange/play.py", line 269, in <module> main() File "habitat_extensions/tasks/rearrange/play.py", line 219, in main env: RearrangeRLEnv = env_cls(config) File "/home/ubuntu/work/hab-mobile-manipulation/habitat_extensions/tasks/rearrange/env.py", line 31, in __init__ super().__init__(self._core_env_config, dataset=dataset) File "/home/ubuntu/work/hab-mobile-manipulation/habitat-lab/habitat/core/env.py", line 377, in __init__ self._env = Env(config, dataset) File "/home/ubuntu/work/hab-mobile-manipulation/habitat-lab/habitat/core/env.py", line 108, in __init__ id_sim=self._config.SIMULATOR.TYPE, config=self._config.SIMULATOR File "/home/ubuntu/work/hab-mobile-manipulation/habitat-lab/habitat/sims/registration.py", line 19, in make_sim return _sim(**kwargs) File "/home/ubuntu/work/hab-mobile-manipulation/habitat_extensions/tasks/rearrange/sim.py", line 93, in __init__ ARM_URDF, joint_indices=[0, 1, 2, 3, 4, 5, 6], ee_link_idx=7 File "/home/ubuntu/work/hab-mobile-manipulation/habitat_extensions/robots/pybullet_utils.py", line 45, in __init__ physicsClientId=self.client_id,
pybullet.error: Cannot load URDF file.
I'd appreciate it if you could help me.
Best,
Zihao Luan