jkulhanek / robot-visual-navigation Goto Github PK

Visual Navigation in Real-World Indoor Environments Using End-to-End Deep Reinforcement Learning Official Implementation

License: MIT License

Python 89.16% CMake 10.11% HTML 0.09% Shell 0.64%

robot-visual-navigation's Introduction

Visual Navigation in Real-World Indoor Environments Using End-to-End Deep Reinforcement Learning

This repository contains the official implementation of paper Visual Navigation in Real-World Indoor Environments Using End-to-End Deep Reinforcement Learning.

Paper Web Demo

Getting started

Before getting started, ensure, that you have Python 3.6+ ready. We recommend activating a new virtual environment for the repository:

python -m venv robot-visual-navigation-env
source robot-visual-navigation-env/bin/activate

Start by cloning this repository and installing the dependencies:

git clone https://github.com/jkulhanek/robot-visual-navigation.git
cd robot-visual-navigation
pip install -r requirements.txt
cd python

For DMHouse package, we recommend starting with Ubuntu 18+ and installing dependencies as follows:

apt-get install libsdl2-dev libosmesa6-dev gettext g++ unzip zip curl gnupg libstdc++6

Downloading the trained models and datasets

You can download the pre-trained models from: https://data.ciirc.cvut.cz/public/projects/2021RealWorldNavigation/checkpoints/dmhouse-models.tar.gz https://data.ciirc.cvut.cz/public/projects/2021RealWorldNavigation/checkpoints/turtlebot-models.tar.gz

Download the pre-trained models using the following commands:

mkdir -p ~/.cache/robot-visual-navigation/models

# Download DMHouse models
curl -L https://data.ciirc.cvut.cz/public/projects/2021RealWorldNavigation/checkpoints/dmhouse-models.tar.gz | tar -xz -C ~/.cache/robot-visual-navigation/models

# Download real-world dataset models
curl -L https://data.ciirc.cvut.cz/public/projects/2021RealWorldNavigation/checkpoints/turtlebot-models.tar.gz | tar -xz -C ~/.cache/robot-visual-navigation/models

# Download real-world dataset
mkdir -p ~/.cache/robot-visual-navigation/datasets
curl -L -o ~/.cache/robot-visual-navigation/datasets/turtle_room_compiled.hdf5 https://data.ciirc.cvut.cz/public/projects/2021RealWorldNavigation/datasets/turtle_room_compiled.hdf5

Evaluation

Run the evaluation on the DMHouse simulator to verify that everything is working correctly:

python evaluate_dmhouse.py dmhouse --num-episodes 100

Similarly for the real-world dataset:

python evaluate_turtlebot.py turtlebot --num-episodes 100

Alternatively, you can also use other agents as described in the Training section.

Training

Start the training by running ./train.py <trainer>, where trainer is the experiment you want to run. Available experiments are the following:

dmhouse: our method (A2CAT-VN) trained with the dmhouse simulator
dmhouse-unreal: UNREAL trained with the dmhouse simulator
dmhouse-a2c: PAAC trained with the dmhouse simulator
turtlebot: our method (A2CAT-VN) fine-tuned on the real-world dataset
turtlebot-unreal: UNREAL fine-tuned on the real-world dataset
turtlebot-a2c: PAAC fine-tuned on the real-world dataset
turtlebot-noprior: our method (A2CAT-VN) trained on the real-world dataset; the model is trained from scretch
turtlebot-unreal-noprior: UNREAL trained on the real-world dataset; the model is trained from scretch
turtlebot-a2c-noprior: PAAC trained on the real-world dataset; the model is trained from scretch

Model checkpoints

All model checkpoints are available online:
https://data.ciirc.cvut.cz/public/projects/2021RealWorldNavigation/checkpoints

Citation

Please use the following citation:

@article{kulhanek2021visual,
  title={Visual navigation in real-world indoor environments using end-to-end deep reinforcement learning},
  author={Kulh{\'a}nek, Jon{\'a}{\v{s}} and Derner, Erik and Babu{\v{s}}ka, Robert},
  journal={IEEE Robotics and Automation Letters},
  volume={6},
  number={3},
  pages={4345--4352},
  year={2021},
  publisher={IEEE}
}

robot-visual-navigation's People

Contributors

Stargazers

Watchers

Forkers

leefree-git h-fu zhefan-xu monicapina rokhmatf pineapplepizza233 sshaayan kotthoff yanxg alexseysua hanxiao485

robot-visual-navigation's Issues

model migration

When I read the paper, I found a problem: the pre-trained simulation environment in the paper is very similar to the real environment layout used, but when we transplant the trained model to the real robot, can the real environment be inconsistent with the distribution of objects in the simulated environment?
Waiting for your answer, thank you very much

Dataset problem

Hi! After having installed all the necessary requirements to be able to use the repository on my pc, I have tried to run the code as described in the help files. However I get this error that seems to be related to a dataset , and I can't find a way to fix it.

Thanks in advance.

train problem

after I successfully installed the DeepMind Lab package, dmhouse and Deep RL PyTorch, then I ran the command python train.py dmhouse

however I found an error and I don't know how to handle it.

Traceback (most recent call last):
  File "train.py", line 41, in <module>
    import trainer as _  # noqa:F401
  File "/home/rokhmat/tesis/robot-visual-navigation/python/trainer.py", line 6, in <module>
    from deep_rl.common.env import RewardCollector, TransposeImage
  File "/home/rokhmat/tesis/robot-visual-navigation/deep-rl-pytorch/deep_rl/common/env.py", line 170, in <module>
    class VecTransposeImage(gym.vector.vector_env.VectorEnvWrapper):
AttributeError: module 'gym.vector.vector_env' has no attribute 'VectorEnvWrapper'

in another PC and virtual env i tried to install an older version of gym and run the command as above i.e. python train.py dmhouse, but i got a different error as below

(tesis) rokhmat@b401:~/tesis/robot-visual-navigation/python$ python train.py dmhouse
Registering trainer turtlebot
Registering trainer turtlebot-end
Registering trainer turtlebot-noprior
Registering trainer turtlebot-unreal
Registering trainer turtlebot-unreal-noprior
Registering trainer turtlebot-a2c
Registering trainer turtlebot-a2c-noprior
Registering trainer dmhouse-a2c
Registering trainer dmhouse-unreal
Registering trainer dmhouse-ppo
Registering trainer dmhouse-ppo-a2cvn
Registering trainer turtlebot-ppo-a2cvn
Registering trainer dmhouse-ppo-unreal
Registering trainer dmhouse-dqn
Registering agent dmhouse-dqn
Registering trainer dmhouse
Registering agent turtlebot-noprior
Registering agent turtlebot-end
Registering agent turtlebot
Registering agent dmhouse
Registering agent dmhouse-a2c
Registering agent dmhouse-ppo
Registering agent dmhouse-unreal
Registering agent turtlebot-a2c
Registering agent turtlebot-unreal
Registering agent turtlebot-a2c-noprior
Registering agent turtlebot-unreal-noprior
starting dmhouse
setting iterationsWithSameMap to 50
setting iterationsWithSameMap to 50
setting iterationsWithSameMap to 50
setting iterationsWithSameMap to 50
setting iterationsWithSameMap to 50
setting iterationsWithSameMap to 50
setting iterationsWithSameMap to 50
setting iterationsWithSameMap to 50
setting iterationsWithSameMap to 50
setting iterationsWithSameMap to 50
setting iterationsWithSameMap to 50
setting iterationsWithSameMap to 50
setting iterationsWithSameMap to 50
setting iterationsWithSameMap to 50
setting iterationsWithSameMap to 50
setting iterationsWithSameMap to 50
setting iterationsWithSameMap to 50
setting iterationsWithSameMap to 50
setting iterationsWithSameMap to 50
================================================================
Total params: 4,892,691
Trainable params: 4,892,691
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 0.07
Params size (MB): 18.66
================================================================
Using CPU only
ERROR: Received the following error from Worker-2: ValueError: could not broadcast input array from shape (21168,) into shape (28224,)
ERROR: Shutting down Worker-2.
ERROR: Received the following error from Worker-1: ValueError: could not broadcast input array from shape (21168,) into shape (28224,)
ERROR: Shutting down Worker-1.
ERROR: Received the following error from Worker-0: ValueError: could not broadcast input array from shape (21168,) into shape (28224,)
ERROR: Shutting down Worker-0.
ERROR: Received the following error from Worker-5: ValueError: could not broadcast input array from shape (21168,) into shape (28224,)
ERROR: Shutting down Worker-5.
ERROR: Received the following error from Worker-9: ValueError: could not broadcast input array from shape (21168,) into shape (28224,)
ERROR: Shutting down Worker-9.
ERROR: Received the following error from Worker-12: ValueError: could not broadcast input array from shape (21168,) into shape (28224,)
ERROR: Shutting down Worker-12.
ERROR: Received the following error from Worker-13: ValueError: could not broadcast input array from shape (21168,) into shape (28224,)
ERROR: Shutting down Worker-13.
ERROR: Received the following error from Worker-10: ValueError: could not broadcast input array from shape (21168,) into shape (28224,)
ERROR: Shutting down Worker-10.
ERROR: Received the following error from Worker-4: ValueError: could not broadcast input array from shape (21168,) into shape (28224,)
ERROR: Shutting down Worker-4.
ERROR: Received the following error from Worker-11: ValueError: could not broadcast input array from shape (21168,) into shape (28224,)
ERROR: Shutting down Worker-11.
ERROR: Received the following error from Worker-14: ValueError: could not broadcast input array from shape (21168,) into shape (28224,)
ERROR: Shutting down Worker-14.
ERROR: Received the following error from Worker-3: ValueError: could not broadcast input array from shape (21168,) into shape (28224,)
ERROR: Shutting down Worker-3.
ERROR: Received the following error from Worker-15: ValueError: could not broadcast input array from shape (21168,) into shape (28224,)
ERROR: Shutting down Worker-15.
ERROR: Received the following error from Worker-7: ValueError: could not broadcast input array from shape (21168,) into shape (28224,)
ERROR: Shutting down Worker-7.
ERROR: Received the following error from Worker-6: ValueError: could not broadcast input array from shape (21168,) into shape (28224,)
ERROR: Shutting down Worker-6.
ERROR: Received the following error from Worker-8: ValueError: could not broadcast input array from shape (21168,) into shape (28224,)
ERROR: Shutting down Worker-8.
ERROR: Raising the last exception back to the main process.
Traceback (most recent call last):
  File "train.py", line 49, in <module>
    trainer.run()
  File "/home/rokhmat/tesis/deep-rl-pytorch/deep_rl/core.py", line 219, in run
    return self.trainer.run(self.process)
  File "/home/rokhmat/tesis/deep-rl-pytorch/deep_rl/common/train_wrappers.py", line 47, in run
    ret = super().run(*args, **kwargs)
  File "/home/rokhmat/tesis/deep-rl-pytorch/deep_rl/core.py", line 207, in run
    return self.trainer.run(process, **kwargs)
  File "/home/rokhmat/tesis/deep-rl-pytorch/deep_rl/core.py", line 207, in run
    return self.trainer.run(process, **kwargs)
  File "/home/rokhmat/tesis/deep-rl-pytorch/deep_rl/common/train_wrappers.py", line 126, in run
    return super().run(_late_process, **kwargs)
  File "/home/rokhmat/tesis/deep-rl-pytorch/deep_rl/core.py", line 207, in run
    return self.trainer.run(process, **kwargs)
  File "/home/rokhmat/tesis/deep-rl-pytorch/deep_rl/core.py", line 239, in run
    super().run(process, **kwargs)
  File "/home/rokhmat/tesis/deep-rl-pytorch/deep_rl/core.py", line 180, in run
    self.model = self._initialize(**self._model_kwargs)
  File "/home/rokhmat/tesis/deep-rl-pytorch/deep_rl/actor_critic/unreal/unreal.py", line 260, in _initialize
    self.rollouts = RolloutStorage(self.env.reset(), self._initial_states(self.num_processes))
  File "/home/rokhmat/anaconda3/envs/tesis/lib/python3.8/site-packages/gym/vector/vector_env.py", line 80, in reset
    return self.reset_wait(seed=seed, return_info=return_info, options=options)
  File "/home/rokhmat/anaconda3/envs/tesis/lib/python3.8/site-packages/gym/vector/async_vector_env.py", line 308, in reset_wait
    self._raise_if_errors(successes)
  File "/home/rokhmat/anaconda3/envs/tesis/lib/python3.8/site-packages/gym/vector/async_vector_env.py", line 627, in _raise_if_errors
    raise exctype(value)
ValueError: could not broadcast input array from shape (21168,) into shape (28224,)

Is there any suggestion that can solve the error I'm having?
Thanks for your attention. I’m looking forward to your reply.

question about ROS workspace

Hello, I find the part about ros in your code was submitted two years ago, some of the steps are vague to me, such as the part I circled with a red box in the figure below, especially the two weights.pth files acquisition. I do not find the weights.pth of the turtlebot-end model in the downloaded models. Could you please update the readme.txt inside this ros folder to be more clear if possible? thank you

about image transform

for the function observation:
the pixel value of input image numpy array ranges from 0 to 255,
and pixel value of output image ranges from -3.332 to 1.593, its not from -1.0 to 1.0, why?
but there is the gym.space.Box(-1.0, 1.0)

puzzle about the construction of occupancy grid map

What algorithm is used for the occupancy grid map in the paper? gmapping, hector-slam or else?
perhaps this is not the key point, but it puzzles me.

[question] training process

Hi!,
when the training process was running then my computer crashed at about 2 million steps and it resulted in a restart on the computer, can the training process continue at the last step when the computer crashes or have to start from the beginning?

[question] collect real world dataset for training

1. collect real world dataset

there are two approach to do the task

1). following the instruction in Collecting the original dataset for training in readme.md

rosrun map_collector controller.py
--by 1 2
--by 2 2
--by 3 2
--goal 4 2

The AGV can only collect images in the setting path and only in the forward angle.

here is the running result:

2). explore mode in map_collector ROS folder
when we run explore function in ros/map_collector/src/main.py, the AGV move from (0,0) to (1,0), then (2,0), finally it come back (0,0) in old way. It can collect pictures in several angles in each positon.

here is the running result:

2. training (python train.py turtlebot)

we plan to use {name}_compiled.hdf5 from 1 to train our own model
as for agent in Reinforcement learning, each positon in hdf5 dataset must have 4 angles(0, 1, 2, 3 seperately)

In the turtle_room_compiled.hdf5 file, the dimention size 20 x 10 x 4 x 10 means each of 4 angles can store 10 photos.

So, our question is which approach is correct for training?
we think the setting path way 1) does not work in Reinforcement learning during traing.
While, the explore method 2) when we tested can not collect the whole indoor playground.
we need collect all the images and infomations of the ground positions which AGV can visit. Is the idea true?

problem about transplantation

it is what happened when I run the command on Turtlebot2：python evaluate_dmhouse.py dmhouse --num-episodes 100
but in fact deepmind_lab.so exists in the corresponding path.
this problem puzzles me for about one week.

puzzle about training result

Is this training result normal? espacially for the result of "turtlebot" and "turtlebot-a2c"

Collecting the original dataset

collect dataset
When I run the command to collect the dataset based on the readme
roslaunch map_collector/launch/start.launch

rosrun map_collector controller.py --by 1 2 --by 2 2 --by 3 2 --goal 4 2

I'm having some problems, the robot moves after running it, but it doesn't create a dataset folder under the /mnt/data/ folder, no images are collected.

train

Author, I'm getting an error when I'm running train.

usage: train [-h] [--no-wandb] trainer
train: error: the following arguments are required: trainer

training problems

when l train model in real world dataset,l meet some error and the final result is differ from the paper.
this is the error during the training

and this is the training result

puzzle about 他和constructionoccupancy grid map

problem about dataset

are there public pre-training datasets now? if there are, how can we get it?

Dmhouse error

Hello, I have the following error when running train.py dmhouse. How can I solve it。

render_video problem

I made an issue based on answer from #4 (comment) and i still can't run command python render_video.py , my terminal output like below

/home/rokhmat/anaconda3/envs/tesis/lib/python3.8/site-packages/deep_rl/common/util.py:1: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
  from collections import OrderedDict, Callable
Registering trainer turtlebot
Registering trainer turtlebot-noprior
Registering trainer turtlebot-unreal
Registering trainer turtlebot-unreal-noprior
Registering trainer turtlebot-a2c
Registering trainer turtlebot-a2c-noprior
Registering trainer dmhouse-a2c
Registering trainer dmhouse-unreal
Registering trainer dmhouse-ppo
Registering trainer dmhouse-ppo-a2cvn
Registering trainer turtlebot-ppo-a2cvn
Registering trainer dmhouse-ppo-unreal
Registering trainer dmhouse-dqn
Registering agent dmhouse-dqn
Registering trainer dmhouse
Registering agent turtlebot-noprior
Registering agent turtlebot
Registering agent dmhouse
Registering agent dmhouse-a2c
Registering agent dmhouse-ppo
Registering agent dmhouse-unreal
Registering agent turtlebot-a2c
Registering agent turtlebot-unreal
Registering agent turtlebot-a2c-noprior
Registering agent turtlebot-unreal-noprior
Traceback (most recent call last):
  File "render_video.py", line 76, in <module>
    env = trainer.create_envs(1, dict(renderer="hardware", screen_size=(500, 500), **default_args()['env_kwargs']),
  File "/home/rokhmat/tesis/robot-visual-navigation/python/trainer.py", line 550, in create_envs
    env = create_multiscene(num_training_processes, wrap=wrap_internal, use_dummy=use_dummy, **env_kwargs)
  File "/home/rokhmat/tesis/robot-visual-navigation/python/environment/__init__.py", line 58, in create_multiscene
    return AsyncVectorEnv(funcs)
  File "/home/rokhmat/anaconda3/envs/tesis/lib/python3.8/site-packages/gym/vector/async_vector_env.py", line 128, in __init__
    dummy_env = env_fns[0]()
  File "/home/rokhmat/tesis/robot-visual-navigation/python/environment/__init__.py", line 49, in func
    env = wrap(gym.make(**kwargs))
  File "/home/rokhmat/anaconda3/envs/tesis/lib/python3.8/site-packages/gym/envs/registration.py", line 676, in make
    return registry.make(id, **kwargs)
  File "/home/rokhmat/anaconda3/envs/tesis/lib/python3.8/site-packages/gym/envs/registration.py", line 520, in make
    return spec.make(**kwargs)
  File "/home/rokhmat/anaconda3/envs/tesis/lib/python3.8/site-packages/gym/envs/registration.py", line 137, in make
    env = self.entry_point(**_kwargs)
  File "/home/rokhmat/tesis/robot-visual-navigation/python/environment/__init__.py", line 12, in _createImageEnvironment
    return ImageEnvironmentWrapper(TimeLimit(ImageEnvironment(**kwargs), 300))
  File "/home/rokhmat/tesis/robot-visual-navigation/python/environment/image_collection_environment.py", line 45, in __init__
    super(ImageEnvironment, self).__init__(**kwargs)
TypeError: object.__init__() takes exactly one argument (the instance to initialize)

is there something missing from the command I ran? or are there steps i have to do?
Previously I have downloaded the pre-trained models that you provide.
Thanks for your attention. I’m looking forward to your reply.

A question about running goal_picker main.py

When I run roslaunch robot_controller start.launch ,I encounter the following error.So I ran the commands in the launch file one by one and found that I had problems running rosrun goal_picker main.py

rosrun goal_picker main.py
Goal picker node connected

Serving Flask app "main" (lazy loading)
Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
Debug mode: on
Goal picker node connected
shutdown request: [/goal_picker] Reason: new node registered with same name

Does it affect navigation? How can I fix it? Thank you for your help.

[Problem] An error caused by inputting camera and target image to pytorch model in real world mode in ros folder

We try to run the real world part of the open source code. While, an error occured when running the code described in the part of the [Using the trained model for the final navigation] in README file(we follow the instruction in your README.md in ~/Documents/AGV_visualnav_dev/robot-visual-navigation/ros/)

The launch file to run:

As we write in the launch, in order to run the goal visual navigation in the real world, we run 3 parts of the codes in the ros folder, which are as follows:

main.py in ros/src/ros_agent_service folder
main.py in ros/src/robot_controller folder
set_goal.py in ros/src/goal_picker folder, whose argument is the goal image

The error information in the terminal output:

(visualnav) agv01@agv01:~/Documents/AGV_visualnav_dev/robot-goal-visual-navigation/goalvisualnav_ros$ bash goalviusalnav.sh 
... logging to /home/agv01/.ros/log/a5a32776-291a-11ed-8ba0-02421e8cbe4b/roslaunch-agv01-32765.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://agv01:45733/

SUMMARY
========

PARAMETERS
 * /part2/image: /home/agv01/Docu...
 * /rosdistro: melodic
 * /rosversion: 1.14.13

NODES
  /part1/
    agent_service (ros_agent_service/main.py)
    controller (robot_controller/main.py)
  /part2/
    goal_pick (goal_picker/setgoal.py)

ROS_MASTER_URI=http://localhost:11311

process[part1/agent_service-1]: started with pid [321]
process[part1/controller-2]: started with pid [322]
process[part2/goal_pick-3]: started with pid [323]
[INFO] [1661943087.874406]: starting is sdu
Controller init success
Puller init success

…………………………..
…………………………..
…………………………..
[INFO] [1661943191.246993]: running
[INFO] [1661943191.253674]: waiting for target
/home/agv01/.local/lib/python3.6/site-packages/gym/core.py:26: UserWarning: WARN: Gym minimally supports python 3.6 as the python foundation not longer supports the version, please update your version to 3.7+
  "Gym minimally supports python 3.6 as the python foundation not longer supports the version, please update your version to 3.7+"
ros_agent_service: starting
model: turtlebot
/home/agv01/Documents/AGV_visualnav_dev/robot-goal-visual-navigation/python/model.py:1: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
Registering trainer turtlebot
Registering trainer turtlebot-noprior
Registering trainer turtlebot-unreal
Registering trainer turtlebot-unreal-noprior
Registering trainer turtlebot-a2c
Registering trainer turtlebot-a2c-noprior
Registering trainer dmhouse-a2c
Registering trainer dmhouse-unreal
Registering trainer dmhouse
Registering agent turtlebot
Registering agent turtlebot-noprior
Registering agent dmhouse
Registering agent dmhouse-a2c
Registering agent dmhouse-unreal
Registering agent turtlebot-a2c
Registering agent turtlebot-unreal
Registering agent turtlebot-a2c-noprior
Registering agent turtlebot-unreal-noprior
robot_visual_navigation/begin_compute_step: subscriber started
robot_visual_navigation/reset_state: subscriber started
Published
target got!
state has been cleared
[INFO] [1661943215.568396]: waiting for observation
[INFO] [1661943215.579759]: waiting for action
begin_compute_step: received request
observation to numpy
dtype_class <class 'numpy.uint8'>
dtype:  uint8
dtype:  uint8
/home/agv01/Documents/AGV_visualnav_dev/robot-goal-visual-navigation/goalvisualnav_ros/src/visualnav_ros_agent_service/src/convert.py:78: DeprecationWarning: The binary mode of fromstring is deprecated, as it behaves surprisingly on unicode inputs. Use frombuffer instead
  data = np.fromstring(msg.data, dtype=dtype).reshape(shape)
data.shape:  (360, 640, 3)
data.shape:  (360, 640, 3)
goal to numpy
dtype_class <class 'numpy.uint8'>
dtype:  uint8
dtype:  uint8
(360, 640, 3)
observation shape:  (360, 640, 3)
observation shape = (84, 84, 3)
observation size = 21168
dtype = uint8
goal shape = (84, 84, 3)
goal size = 21168
goal dtype = uint8

[ERROR] [1661934068.494090]: bad callback: <function create_send_response.<locals>.send_response at 0x7f5231a7b8>
Traceback (most recent call last):
  File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/topics.py", line 750, in _invoke_callback
    cb(msg)
  File "/home/agv01/Documents/AGV_visualnav_dev/robot-goal-visual-navigation/goalvisualnav_ros/src/visualnav_ros_agent_service/src/main.py", line 63, in send_response
    action = server.compute_step(request)
  File "/home/agv01/Documents/AGV_visualnav_dev/robot-goal-visual-navigation/goalvisualnav_ros/src/visualnav_ros_agent_service/src/main.py", line 52, in compute_step
    action = self.agent.act((observation, goal,))
  File "/home/agv01/Documents/AGV_visualnav_dev/robot-goal-visual-navigation/python/agent.py", line 36, in act
    action = old_act(obs)
  File "/home/agv01/Documents/AGV_visualnav_dev/robot-goal-visual-navigation/python/trainer.py", line 353, in act
    action, self.states = self._step(obs, self.states)
  File "/home/agv01/Documents/AGV_visualnav_dev/robot-goal-visual-navigation/python/deep_rl/common/pytorch.py", line 62, in call
    to_tensor(kwargs, device))
  File "/home/agv01/Documents/AGV_visualnav_dev/robot-goal-visual-navigation/python/trainer.py", line 342, in step
    policy_logits, _, states = model(observations, masks, states)
  File "/home/agv01/miniforge-pypy3/envs/visualnav/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/agv01/Documents/AGV_visualnav_dev/robot-goal-visual-navigation/python/model.py", line 118, in forward
    features, states = self._forward_base(inputs, masks, states)
  File "/home/agv01/Documents/AGV_visualnav_dev/robot-goal-visual-navigation/python/model.py", line 149, in _forward_base
    return self.rnn(features, masks, states)
  File "/home/agv01/miniforge-pypy3/envs/visualnav/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/agv01/Documents/AGV_visualnav_dev/robot-goal-visual-navigation/python/deep_rl/model/module.py", line 38, in forward
    return forward_masked_rnn_transposed(inputs, masks, states, self.inner)
  File "/home/agv01/Documents/AGV_visualnav_dev/robot-goal-visual-navigation/python/deep_rl/common/pytorch.py", line 221, in forward_masked_rnn_transposed
    outputs, states = forward_masked_rnn(inputs, masks, states, forward_rnn)
  File "/home/agv01/Documents/AGV_visualnav_dev/robot-goal-visual-navigation/python/deep_rl/common/pytorch.py", line 113, in forward_masked_rnn
    mask_states(states, masks[:, start_idx])
  File "/home/agv01/miniforge-pypy3/envs/visualnav/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/agv01/miniforge-pypy3/envs/visualnav/lib/python3.6/site-packages/torch/nn/modules/rnn.py", line 689, in forward
    self.check_forward_args(input, hx, batch_sizes)
  File "/home/agv01/miniforge-pypy3/envs/visualnav/lib/python3.6/site-packages/torch/nn/modules/rnn.py", line 632, in check_forward_args
    self.check_input(input, batch_sizes)
  File "/home/agv01/miniforge-pypy3/envs/visualnav/lib/python3.6/site-packages/torch/nn/modules/rnn.py", line 207, in check_input
    self.input_size, input.size(-1)))
RuntimeError: input.size(-1) must be equal to input_size. Expected 518, got 517

[turtlesim2/goal_pick-3] process has finished cleanly
log file: /home/agv01/.ros/log/b475da3a-2903-11ed-8885-02421e8cbe4b/turtlesim2-goal_pick-3*.log
^C[turtlesim1/controller-2] killing on exit
[turtlesim1/agent_service-1] killing on exit
[INFO] [1661935536.123854]: received action None
action:  None
shutting down processing monitor...
... shutting down processing monitor complete
Done

So the error is

[ERROR] [1661934068.494090]: bad callback: <function create_send_response.<locals>.send_response at 0x7f5231a7b8>
......
......
Traceback (most recent call last):`RuntimeError: input.size(-1) must be equal to input_size. Expected 518, got 517

Error Analysis:
The target image and observation image saved in the function compute_step during running in ~/Documents/AGV_visualnav_dev/robot-visual-navigation/ros/src/ros_agent_service/src/main.py, so we guess the pictures are suitable with the codes and there are no business between the pictures and the error.

Potentially relevant error codes located in the model.py (~/Documents/AGV_visualnav_dev/robot-goal-visual-navigation/python/model.py)

Running environment:

Hardware description:
use Nvidia Jetson AGX Xavier and an AGV to run the code in ros folder

Software description:
Ubuntu 18.04
Use miniforge(conda) to create python3.6 environment (/home/agv01/miniforge-pypy3/envs/visualnav/bin/python3.6)
Pytorch version 1.10

The illustration of the model in the paper:

The question is whether the error is caused by our data format in captured image or the RNN / LSTM model code in pytorch provided in ros folder ?
or, in order to run the real world part, are the codes we run correct ?

Thanks very much.

ROS1 or ROS2

I want to know which version of ros your turtlebot used? ROS1 is installed on my robot. ROS1 supports python2.7 by default, but the readme file shows that the environment requires python3.6+. How did you solve this problem when porting the trained model to a real robot?

error about sampling data from storage

If I use trainer "turtlebot" or "turtlebot-unreal" to train， errors occurred like below and then cause the reward of them dont normal(too low):

install issues

after l successfully install the DeepMind Lab,l begin to Install the deeprl package,then,l meet a error ,l don't how to deal it .

ERROR: Command errored out with exit status 1:
command: /home/kangwen/anaconda3/envs/rvn/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/kangwen/robot-visual-navigation/deep-rl-pytorch/setup.py'"'"'; file='"'"'/home/kangwen/robot-visual-navigation/deep-rl-pytorch/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-p0e_oamq
cwd: /home/kangwen/robot-visual-navigation/deep-rl-pytorch/
Complete output (15 lines):
Traceback (most recent call last):
File "", line 1, in
File "/home/kangwen/robot-visual-navigation/deep-rl-pytorch/setup.py", line 2, in
from deep_rl import version
File "/home/kangwen/robot-visual-navigation/deep-rl-pytorch/deep_rl/init.py", line 1, in
from .registry import make_agent, make_trainer, register_agent, register_trainer
File "/home/kangwen/robot-visual-navigation/deep-rl-pytorch/deep_rl/registry.py", line 1, in
from .common import train_wrappers as wrappers
File "/home/kangwen/robot-visual-navigation/deep-rl-pytorch/deep_rl/common/init.py", line 1, in
from .train_wrappers import MetricContext
File "/home/kangwen/robot-visual-navigation/deep-rl-pytorch/deep_rl/common/train_wrappers.py", line 5, in
from ..core import AbstractTrainerWrapper, MetricContext
File "/home/kangwen/robot-visual-navigation/deep-rl-pytorch/deep_rl/core.py", line 4, in
import gym
ModuleNotFoundError: No module named 'gym'
----------------------------------------
WARNING: Discarding file:///home/kangwen/robot-visual-navigation/deep-rl-pytorch. Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

real robot issues

I am a graduate student and very interested in your research on visual navigation. Recently, I've been having some problems reproducing the results of the real turtlebot2. I ran into some issues when running the code using the kinect1 camera and would really like your guidance and help.

Run the following commands
roslaunch robot_controller start.launch

python src/ros_agent_service/src/main.py

python goal_picker/src/setgoal.py --image /home/zwz/robot-visual-navigation-master/goal/door.jpg
The following error was encountered：

[ERROR] [1704981429.247813]: bad callback: <function create_send_response.<locals>.send_response at 0x7f0fe4461040>
Traceback (most recent call last):
  File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/topics.py", line 750, in _invoke_callback
    cb(msg)
  File "ros_agent_service/src/main.py", line 33, in send_response
    action = server.compute_step(request)
  File "ros_agent_service/src/main.py", line 22, in compute_step
    action = self.agent.act((observation, goal,))
  File "/home/zwz/robot-visual-navigation-master/ros/src/ros_agent_service/src/agent.py", line 57, in act
    obs = (_transform_image(img), _transform_image(goal))
  File "/home/zwz/robot-visual-navigation-master/ros/src/ros_agent_service/src/agent.py", line 13, in _transform_image
    return np.expand_dims(((img.astype(np.float32) - mean) / std).transpose([2, 0, 1]), 0)
ValueError: axes don't match array

img.shape (84, 84) [[ 46  51  47 ... 110 107 108]
 [ 53  52  26 ... 125  98 106]
 [ 58  42  46 ... 124 117 116]
 ...
 [101  95  98 ... 128 129 128]
 [ 96 106  86 ... 123 129 126]
 [ 91  95  72 ... 123 129 125]]

goal.shape (84, 84, 3) [[[139 158 154]
  [139 158 154]
  [144 162 161]
  ...
  [150 170 169]
  [147 167 166]
  [145 165 166]]

 [[139 158 154]
  [141 160 156]
  [144 162 161]
  ...
  [152 170 172]
  [149 167 169]
  [147 165 165]]

 [[139 158 154]
  [142 161 157]
  [143 162 160]
  ...
  [155 170 176]
  [152 167 170]
  [149 165 163]]

According to the printout, the goal image is a 3-channel RGB image and the observation image is a single-channel grayscale image.
I found the gmapping_demo.launch and 3dsensor.launch for starting the sensor based on the command
roslaunch robot_controller start.launch
how to start the RGB image in these two files, and what should I change in the specific code, looking forward to your reply.

gmapping_demo.launch

<launch>
  <!-- 3D sensor -->
  <arg name="3d_sensor" default="$(env TURTLEBOT_3D_SENSOR)"/>  <!-- r200, kinect, asus_xtion_pro -->
  <include file="$(find turtlebot_bringup)/launch/3dsensor.launch">
    <arg name="rgb_processing" value="false" />
    <arg name="depth_registration" value="false" />
    <arg name="depth_processing" value="false" />
    
    <!-- We must specify an absolute topic name because if not it will be prefixed by "$(arg camera)".
         Probably is a bug in the nodelet manager: https://github.com/ros/nodelet_core/issues/7 --> 
    <arg name="scan_topic" value="/scan" />
  </include>

  <!-- Gmapping -->
  <arg name="custom_gmapping_launch_file" default="$(find turtlebot_navigation)/launch/includes/gmapping/$(arg 3d_sensor)_gmapping.launch.xml"/>
  <include file="$(arg custom_gmapping_launch_file)"/>

  <!-- Move base -->
  <include file="$(find turtlebot_navigation)/launch/includes/move_base.launch.xml"/>

</launch>

3dsensor.launch

<!-- 
  Turtlebot is a bit low on horsepower...

  We use openni_camera here, turn everything on by default
  (allows the user to conveniently see everything when
  launching this on its own - use with 
  turtebot_rviz_launchers/view_robot.launch to visualise)
  and provide args to disable them so you can optimise the
  horsepower for your application.
  
  For an example of disabling processing modules, check
  any of the turtlebot_rapps (e.g. android_make_a_map.launch
  only enables scan_processing for depthimage_to_laserscan and
  rgb_processing for the android tele-view).
-->
<launch>
  <!-- "camera" should uniquely identify the device. All topics are pushed down
       into the "camera" namespace, and it is prepended to tf frame ids. -->
  <arg name="camera"      default="camera"/>
  <arg name="publish_tf"  default="false"/>
  <arg name="3d_sensor"   default="$(env TURTLEBOT_3D_SENSOR)"/>  <!-- kinect, asus_xtion_pro -->

  <!-- Factory-calibrated depth registration -->
  <arg name="depth_registration"              default="true"/>
  <arg     if="$(arg depth_registration)" name="depth" value="depth_registered" />
  <arg unless="$(arg depth_registration)" name="depth" value="depth" />

  <!-- Processing Modules -->
  <arg name="rgb_processing"                  default="true"/>
  <arg name="ir_processing"                   default="true"/>
  <arg name="depth_processing"                default="true"/>
  <arg name="depth_registered_processing"     default="true"/>
  <arg name="disparity_processing"            default="true"/>
  <arg name="disparity_registered_processing" default="true"/>
  <arg name="scan_processing"                 default="true"/>

  <!-- Worker threads for the nodelet manager -->
  <arg name="num_worker_threads" default="4" />

  <!-- Laserscan topic -->
  <arg name="scan_topic" default="scan"/>

  <include file="$(find turtlebot_bringup)/launch/includes/3dsensor/$(arg 3d_sensor).launch.xml">
    <arg name="camera"                          value="$(arg camera)"/>
    <arg name="publish_tf"                      value="$(arg publish_tf)"/>
    <arg name="depth_registration"              value="$(arg depth_registration)"/>
    <arg name="num_worker_threads"              value="$(arg num_worker_threads)" />

    <!-- Processing Modules -->
    <arg name="rgb_processing"                  value="$(arg rgb_processing)"/>
    <arg name="ir_processing"                   value="$(arg ir_processing)"/>
    <arg name="depth_processing"                value="$(arg depth_processing)"/>
    <arg name="depth_registered_processing"     value="$(arg depth_registered_processing)"/>
    <arg name="disparity_processing"            value="$(arg disparity_processing)"/>
    <arg name="disparity_registered_processing" value="$(arg disparity_registered_processing)"/>
  </include>

   <!--                        Laserscan 
     This uses lazy subscribing, so will not activate until scan is requested.
   -->
  <group if="$(arg scan_processing)">
    <node pkg="nodelet" type="nodelet" name="depthimage_to_laserscan" args="load depthimage_to_laserscan/DepthImageToLaserScanNodelet $(arg camera)/$(arg camera)_nodelet_manager">
      <!-- Pixel rows to use to generate the laserscan. For each column, the scan will
           return the minimum value for those pixels centered vertically in the image. -->
      <param name="scan_height" value="10"/>
      <param name="output_frame_id" value="$(arg camera)_depth_frame"/>
      <param name="range_min" value="0.45"/>
      <remap from="image" to="$(arg camera)/$(arg depth)/image_raw"/>
      <remap from="scan" to="$(arg scan_topic)"/>

      <!-- Somehow topics here get prefixed by "$(arg camera)" when not inside an app namespace,
           so in this case "$(arg scan_topic)" must provide an absolute topic name (issue #88).
           Probably is a bug in the nodelet manager: https://github.com/ros/nodelet_core/issues/7 -->
      <remap from="$(arg camera)/image" to="$(arg camera)/$(arg depth)/image_raw"/>
      <remap from="$(arg camera)/scan" to="$(arg scan_topic)"/>
    </node>
  </group>
</launch>

Where is the image？

Hello，
I want to use that image where it was in ".../assets/images/63.pngassets/images/63.png". Can you provide it to me ?

Thank you

ValueError when using the latest commit

Hi, I am facing the ValueError when I try to train dmhouse with the code from the latest commit. The error is gone if I revert the commit(2c87a2d).

target_image

What is the input image format when setting the target image on a real robot? rgb image, depth map, or another?
Can you show us the code to get the goal image?

puzzle about coordinates for getting images in real environment

For obtaining images based on coordinates in the real environment, is the value of coordinates the coordinates of points on the map? or something else? That is how to prepare a list of coordinates for the robot to collect images.

ValueError: Not enough values to unpack (expected 4, got 2)

Dear Respect Author:
During training the model I encountered such a value error as the screenshot below:

Indeed, I have read some existed solutions on this problem in the previous issues. Based on the answers @leongkhing said in another question, the error can be fixed after reverting the original code in the file python/deep_rl/a2c_unreal/unreal.py. Unfortunately, it is not worked for my situation.

After this, I just noticed that the batch is a tuple indeed only including two elements that are also tuples, showcasing that it seems not enough for fulfilling four variables. Therefore, could you please help to check are there any other methods to solve this problem, or propose any advise if I got something missed? I will be really grateful for anything you help. Thanks!

[Paper] Clarification on auxiliary tasks

Hello,
Thank you for making your work public. Concerning the paper (Visual Navigation in Real-World Indoor Environments Using
End-to-End Deep Reinforcement Learning), I don't understand well the auxiliary task consisting in reconstructing the input image and the target image, its interest? I understood it as the function materialized by the convolution network must deviate as little as possible from the input image and the target image. I may be totally wrong. Thank you for your answer.