This project assumes that the user has a Robot Operating System (ROS) based simulation and are looking to use it to train a reinforcement learning policy. It is recommended that the user have a basic understanding of ROS or ROS2 before proceeding with the Integrating Your Simulation section.

A fully Bonsai-integrated Turtlebot3 simulation has been provided as an example. See the Turtlebot3 Sample section.

If you are here for the Turtlebot3 Simulator for Bonsai, please go to the Turtlebot3 Bonsai README

If you have your own ROS simulation and want to use it with reinforcement learning, please proceed.

A python connector for integrating Bonsai with robotic simulations that use ROS as a message passing framework. Common simulators for robotics:

• Gazebo
• Webots
• CoppeliaSim
• Unity

This connector is built for ROS2 Foxy Fitzroy.

To use Bonsai, you must provision a Bonsai workspace in an Azure. Your account will need create resources while using Bonsai, so I recommend using a subscription that you have full user access to if possible.

Please see directions for setting up a Microsoft account and a Bonsai workspace.

Once you have selected your workspace in the Bonsai UI, create a new empty brain project.

We will come back to this later once the the simulator is ready to be connected to Bonsai.

A linux workspace or WLS2 is highly reccommended.

1. Install WLS2 (see WSL Installation at Setting up ROS in WLS2)
2. Set up GUI forwarding (see Setting Up GUI Forwarding at Setting up ROS in WLS2)
3. Install ROS2
4. Configure your ROS2 Environment
5. Create a workspace Example
6. Clone this repo into your workspace's src directory
7. In the src directory, clone and rename the package_template directory

The package_template directory contains all of the files necessary to set up a ROS enabled simulation.

Bonsai requires a JSON filed referred to as an interface in order to interact with the simulator. The interface defines state, action, and configuration data. Each type of data is used differently while training a policy.

Sample Interface

Each simulation needs its own interface.

The connector consists of a base, a simulator connection, and a policy connection.

The base is the main setup of the ROS communication that both the simulator and policy will use to pass telemetry.

To begin, you will need a ROS2 workspace. Create a new package and create the following folders: <pacakge_name>/config, <pacakge_name>/launch, and <pacakge_name>/<pacakge_name>. See the turtlebot3 sample for reference.

Copy the python files from src to the <pacakge_name>/<pacakge_name> directory. These files should be edited as follows.

• init(): Topic subscribers and publishers should be initialized here. A common subscription is to /joint_state_data and a common publisher is to /cmd_vel

• init_params(): This function needs to initialize all the state variables (defined in the sim config) to an initialize value, most likely 0.0.

Note that the constructor for the sim_connection class consumes the package where the simulator interface exists and its name.

• init(): For the simulator, additional initialization needs to be done to ensure that the simulation can be reset on command. For Gazebo, this is done through a service call.

• step(): The step function should be developed alongside the simulator interface file's action section. Each action variable corresponds to data that Bonsai will pass to the simulation in order to change its state. Each action will need to be parsed and converted to the requisite ROS message type and published to the topic that will apply the action. For example, changes in velocity should be published to the /cmd_vel topic in the form of a Twist() message.

• reset(): At the end of a simulation, the simulator must be reset in order to continue with a new training episode. The simulator needs to be commanded to reset at the start of this function.

Note that the constructor for the policy_connection class consumes the URL for communicating with the exported Bonsai policy. If the default instructions for exporting and using a policy were followed, this address is http://localhost:5000.

• post_state_data()): This function structures the system (robot) state data into a json message that is then sent to the policy via REST request.

See turtlebot3_bonsai for example launch files.

Policy training instructions are defined in the Bonsai UI space. You will need to go to the empty brain project that you created earlier.

The training configuration is defined using Inkling.

See the Inkling Programming Language Reference for how to plan your policy.

Once you are satisfied with your local simulation, you can create a docker file to deploy your simulation in the cloud. See the turtlebot3_bonsai sample for reference.

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.