This repository aims to make an algoirthm to follow a person autonomously. This algorithm has been successfully tested on a TurtleBot2. Hand gesture for stopping/starting has also added for easier target acquisition. Attempts have been made to reduce the effect of occlusion.

If you are using this repository please cite our paper and star our repo. You can also request the paper on Research Gate or contact authors (typically reply within a day)

Incase you find any problem with the code/repo do let us know by raising an issue.

• Person Following Robot
  • Table of contents
  • Working Demo
  • Results
  • Abstract
  • Project Pipeline
  • Instructions
    • Host Machine (machine running roscore)
    • Remote Machine
    • Run the tracking code
  • Dependencies
    • Hardware
    • Software
  • Citations
  • Contributors

YouTube Link

Helper robots are widely used in various situations, for ex-ample at airports and railway stations. This paper presents a pipeline to multiplex the tracking and detection of a person in dynamic environments using a stereo camera in real-time. Recent developments in object detection using ConvNets have led to robust person detection. These deep convolutional neural networks generally fail to run with high frames rates on devices with less computing power. Trackers are also used to retain the identity of the target person as well as impose fewer constraints on hardware. A concept of multiplexed detection and tracking is used which makes the pipeline faster by many folds. TurtleBot2 is used for prototyping the robot and tuning of the motion controller. Robot Operating System (ROS) is used to set up communication be-tween various nodes of the pipeline. The results found were comparable to current state-of-the-art person followers and can be readily used in day to day life.

The complete pipeline for the person following robot can be understood using the given flow chart.

Setting up remote server for faster processing:

• export ROS_MASTER_URI=http://192.168.0.113:11311 (replace with host machine ip)
• export ROS_IP=192.168.0.123 (replace with host machine ip)
• export ROS_HOSTNAME=192.168.0.123 (replace with host machine ip)

• export ROS_MASTER_URI=http://192.168.0.113:11311 (replace with host machine ip)
• export ROS_IP=192.168.0.123 (replace with local machine ip)
• export ROS_HOSTNAME=192.168.0.123 (replace with host machine ip)

• First install all requirements using pip3 install requirements.txt. Hardware requirements can be found here
• Download Yolov3 weight file from here and place it inside the cloned repo.
• Run the follow.py code to see the output

Its recommended to have a Nvidia GPU for faster processing

• TurtleBot 2 platform
• Intel Realsense Depth Camera
• PC/Onboard Computer(Jetson or )

• Python 3.6+
• OpenCV-python: link
• PyTorch: link
• pyrealsense2: link
• Robot Operating System (ROS): Link

If you find our research useful and want to use it, please cite our paper at

@InProceedings{10.1007/978-981-15-3639-7_98,
author="Agrawal, Khush
and Lal, Rohit",
editor="Kalamkar, Vilas R.
and Monkova, Katarina",
title="Person Following Mobile Robot Using Multiplexed Detection and Tracking",
booktitle="Advances in Mechanical Engineering",
year="2021",
publisher="Springer Singapore",
address="Singapore",
pages="815--822",
isbn="978-981-15-3639-7"
}

• Khush Agrawal - Website
• Rohit Lal - Website