AWS DeepRacer Follow the Leader (FTL) sample project
Overview
The Follow the Leader (FTL) sample project is an sample application built on top of the existing AWS DeepRacer application, which uses an object-detection machine learning model through which the AWS DeepRacer device can identify and follow a person. For detailed information on the Follow the Leader (FTL) sample project, see the Getting Started section.
License
The source code is released under Apache 2.0.
Installation
Follow this procedure for installation.
Prerequisites
The AWS DeepRacer device comes with all the prerequisite packages and libraries installed to run the Follow the Leader (FTL) sample project. For more details about the preinstalled set of packages and libraries on the DeepRacer device, and for instructions for installing the required build systems, see Getting Started with DeepRacer OpenSource. The Follow the Leader (FTL) sample project requires you to install the AWS DeepRacer application on the device, because it leverages most of the packages from the core application.
The following are additional software and hardware requirements for using the Follow the Leader (FTL) sample project on the AWS DeepRacer device.
-
Download and optimize the object-detection model: Follow the instructions to download and optimize the object-detection model and copy it to the required location on the AWS DeepRacer device.
-
Calibrate the AWS DeepRacer (optional): Follow the instructions to calibrate the mechanics of your AWS DeepRacer vehicle so the vehicle performance is optimal and it behaves as expected.
-
Set up the Intel Neural Compute Stick 2 (optional): The
object_detection_nodeprovides functionality to offload the inference to a Intel Neural Compute Stick 2 connected to the AWS DeepRacer device. This is an optional setting that enhances the inference performance of the object-detection model. For more details about running inference on the Movidius NCS (Neural Compute Stick) with OpenVINO™ toolkit, see this video.
Attach the Neural Compute Stick 2 firmly in the back slot of the AWS DeepRacer, open a terminal, and run the following commands as the root user to install the dependencies of the Intel Neural Compute Stick 2 on the AWS DeepRacer device.
-
Switch to the root user:
sudo su -
Navigate to the OpenVino installation directory:
cd /opt/intel/openvino_2021/install_dependencies -
Set the environment variables required to run the Intel OpenVino scripts:
source /opt/intel/openvino_2021/bin/setupvars.sh -
Run the dependency installation script for the Intel Neural Compute Stick:
./install_NCS_udev_rules.sh
Downloading and building
Open a terminal on the DeepRacer device and run the following commands as the root user.
-
Switch to the root user before you source the ROS2 installation:
sudo su -
Stop the deepracer-core.service that is currently running on the device:
systemctl stop deepracer-core -
Source the ROS2 Foxy setup bash script:
source /opt/ros/foxy/setup.bash -
Set the environment variables required to run Intel OpenVino scripts:
source /opt/intel/openvino_2021/bin/setupvars.sh -
Create a workspace directory for the package:
mkdir -p ~/deepracer_ws cd ~/deepracer_ws -
Clone the entire Follow the Leader(FTL) sample project on the DeepRacer device:
git clone https://github.com/aws-deepracer/aws-deepracer-follow-the-leader-sample-project.git cd ~/deepracer_ws/aws-deepracer-follow-the-leader-sample-project/deepracer_follow_the_leader_ws/ -
Clone the
async_web_server_cpp,web_video_server, andrplidar_ros dependencypackages on the DeepRacer device:cd ~/deepracer_ws/aws-deepracer-follow-the-leader-sample-project/deepracer_follow_the_leader_ws/ && ./install_dependencies.sh -
Fetch the unreleased dependencies:
cd ~/deepracer_ws/aws-deepracer-follow-the-leader-sample-project/deepracer_follow_the_leader_ws/ rosws update -
Resolve the dependencies:
cd ~/deepracer_ws/aws-deepracer-follow-the-leader-sample-project/deepracer_follow_the_leader_ws/ && rosdep install -i --from-path . --rosdistro foxy -y -
Build the packages in the workspace:
cd ~/deepracer_ws/aws-deepracer-follow-the-leader-sample-project/deepracer_follow_the_leader_ws/ && colcon build
Using the Follow the Leader (FTL) sample application
Follow this procedure to use the Follow the Leader (FTL) sample application.
Running the node
To launch the Follow the Leader (FTL) sample application as the root user on the AWS DeepRacer device, open another terminal on the device and run the following commands as the root user.
-
Switch to the root user before you source the ROS2 installation:
sudo su -
Source the ROS2 Foxy setup bash script:
source /opt/ros/foxy/setup.bash -
Set the environment variables required to run Intel OpenVino scripts:
source /opt/intel/openvino_2021/bin/setupvars.sh -
Source the setup script for the installed packages:
source ~/deepracer_ws/aws-deepracer-follow-the-leader-sample-project/deepracer_follow_the_leader_ws/install/setup.bash -
Launch the nodes required for the Follow the Leader (FTL) sample project:
ros2 launch ftl_launcher ftl_launcher.py
Enabling followtheleader mode using the CLI
Once the ftl_launcher has been kicked off, open a new terminal as the root user.
-
Switch to the root user before you source the ROS2 installation:
sudo su -
Navigate to the Follow the Leader (FTL) workspace:
cd ~/deepracer_ws/aws-deepracer-follow-the-leader-sample-project/deepracer_follow_the_leader_ws/ -
Source the ROS2 Foxy setup bash script:
source /opt/ros/foxy/setup.bash -
Source the setup script for the installed packages:
source ~/deepracer_ws/aws-deepracer-follow-the-leader-sample-project/deepracer_follow_the_leader_ws/install/setup.bash -
Set the mode of the DeepRacer via
ctrl_pkgtofollowtheleaderusing the following ROS2 service call:ros2 service call /ctrl_pkg/vehicle_state deepracer_interfaces_pkg/srv/ActiveStateSrv "{state: 3}" -
Enable
followtheleadermode using the following ROS2 service call:ros2 service call /ctrl_pkg/enable_state deepracer_interfaces_pkg/srv/EnableStateSrv "{is_active: True}"
Changing the MAX_SPEED scale of the DeepRacer:
You can modify the MAX_SPEED scale of the DeepRacer using a ROS2 service call in case the car isn’t moving as expected. This can occur because of the vehicle battery percentage, the surface on which the car is operating, or for other reasons.
-
Switch to the root user before you source the ROS2 installation:
sudo su -
Navigate to the Follow the Leader(FTL) workspace:
cd ~/deepracer_ws/aws-deepracer-follow-the-leader-sample-project/deepracer_follow_the_leader_ws/ -
Source the ROS2 Foxy setup bash script:
source /opt/ros/foxy/setup.bash -
Source the setup script for the installed packages:
source ~/deepracer_ws/aws-deepracer-follow-the-leader-sample-project/deepracer_follow_the_leader_ws/install/setup.bash -
Change the
MAX SPEEDto xx% of theMAXScale:ros2 service call /ftl_navigation_pkg/set_max_speed deepracer_interfaces_pkg/srv/SetMaxSpeedSrv "{max_speed_pct: 0.xx}"Example: Change the
MAX SPEEDto 75% of theMAXScale:ros2 service call /ftl_navigation_pkg/set_max_speed deepracer_interfaces_pkg/srv/SetMaxSpeedSrv "{max_speed_pct: 0.75}"
Launch files
The ftl_launcher.py included in this package is the main launcher file that launches all the required nodes for the Follow the Leader (FTL) sample project. This launcher file also includes the nodes from the AWS DeepRacer core application.
from launch import LaunchDescription
from launch_ros.actions import Node
def generate_launch_description():
ld = LaunchDescription()
object_detection_node = Node(
package='object_detection_pkg',
namespace='object_detection_pkg',
executable='object_detection_node',
name='object_detection_node',
parameters=[{
'DEVICE': 'CPU',
'PUBLISH_DISPLAY_OUTPUT': True
}]
)
ftl_navigation_node = Node(
package='ftl_navigation_pkg',
namespace='ftl_navigation_pkg',
executable='ftl_navigation_node',
name='ftl_navigation_node'
)
camera_node = Node(
package='camera_pkg',
namespace='camera_pkg',
executable='camera_node',
name='camera_node'
)
ctrl_node = Node(
package='ctrl_pkg',
namespace='ctrl_pkg',
executable='ctrl_node',
name='ctrl_node'
)
deepracer_navigation_node = Node(
package='deepracer_navigation_pkg',
namespace='deepracer_navigation_pkg',
executable='deepracer_navigation_node',
name='deepracer_navigation_node'
)
software_update_node = Node(
package='deepracer_systems_pkg',
namespace='deepracer_systems_pkg',
executable='software_update_node',
name='software_update_node'
)
model_loader_node = Node(
package='deepracer_systems_pkg',
namespace='deepracer_systems_pkg',
executable='model_loader_node',
name='model_loader_node'
)
otg_control_node = Node(
package='deepracer_systems_pkg',
namespace='deepracer_systems_pkg',
executable='otg_control_node',
name='otg_control_node'
)
network_monitor_node = Node(
package='deepracer_systems_pkg',
namespace='deepracer_systems_pkg',
executable='network_monitor_node',
name='network_monitor_node'
)
device_info_node = Node(
package='device_info_pkg',
namespace='device_info_pkg',
executable='device_info_node',
name='device_info_node'
)
battery_node = Node(
package='i2c_pkg',
namespace='i2c_pkg',
executable='battery_node',
name='battery_node'
)
inference_node = Node(
package='inference_pkg',
namespace='inference_pkg',
executable='inference_node',
name='inference_node'
)
model_optimizer_node = Node(
package='model_optimizer_pkg',
namespace='model_optimizer_pkg',
executable='model_optimizer_node',
name='model_optimizer_node'
)
rplidar_node = Node(
package='rplidar_ros',
namespace='rplidar_ros',
executable='rplidarNode',
name='rplidarNode',
parameters=[{
'serial_port': '/dev/ttyUSB0',
'serial_baudrate': 115200,
'frame_id': 'laser',
'inverted': False,
'angle_compensate': True,
}]
)
sensor_fusion_node = Node(
package='sensor_fusion_pkg',
namespace='sensor_fusion_pkg',
executable='sensor_fusion_node',
name='sensor_fusion_node'
)
servo_node = Node(
package='servo_pkg',
namespace='servo_pkg',
executable='servo_node',
name='servo_node'
)
status_led_node = Node(
package='status_led_pkg',
namespace='status_led_pkg',
executable='status_led_node',
name='status_led_node'
)
usb_monitor_node = Node(
package='usb_monitor_pkg',
namespace='usb_monitor_pkg',
executable='usb_monitor_node',
name='usb_monitor_node'
)
webserver_publisher_node = Node(
package='webserver_pkg',
namespace='webserver_pkg',
executable='webserver_publisher_node',
name='webserver_publisher_node'
)
web_video_server_node = Node(
package='web_video_server',
namespace='web_video_server',
executable='web_video_server',
name='web_video_server'
)
ld.add_action(object_detection_node)
ld.add_action(ftl_navigation_node)
ld.add_action(camera_node)
ld.add_action(ctrl_node)
ld.add_action(deepracer_navigation_node)
ld.add_action(software_update_node)
ld.add_action(model_loader_node)
ld.add_action(otg_control_node)
ld.add_action(network_monitor_node)
ld.add_action(device_info_node)
ld.add_action(battery_node)
ld.add_action(inference_node)
ld.add_action(model_optimizer_node)
ld.add_action(rplidar_node)
ld.add_action(sensor_fusion_node)
ld.add_action(servo_node)
ld.add_action(status_led_node)
ld.add_action(usb_monitor_node)
ld.add_action(webserver_publisher_node)
ld.add_action(web_video_server_node)
return ld
Configuration file and parameters
| Parameter Name | Description |
|---|---|
DEVICE (optional) |
If set as MYRIAD, it uses the Intel Compute Stick 2 for inference. Else, uses the CPU for inference by default, even if it is removed. |
PUBLISH_DISPLAY_OUTPUT |
Set to True or False if the inference output images need to be published to localhost using web_video_server. |
Demo
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent