Reinforcement Learning for Spacecraft Docking Trajectories with Proximal Policy Optimization. Learning a model-free approach to unify model switching standard.

Space docking is the process of connecting two spacecraft together in space. It involves bringing the spacecraft into close proximity, aligning their docking ports, and using a docking mechanism to secure the connection. This maneuver enables crew transfers, resupply missions, module attachment, and the formation of larger structures in space. Rendezvous, alignment, capture, sealing, and pressurization are key steps in the docking process. Space docking plays a crucial role in space exploration, satellite servicing, and the assembly of space stations, requiring precise coordination and specialized docking systems to achieve a successful connection in the challenging conditions of space.

Hi!, my name is Vince. I am an undergraduate researcher specializing in control with reinforcement learning (RL). Currently pursuing a Bachelor of Science in Computer Science at the University of California, Irvine, I am passionate about exploring the intersection of machine learning and control systems. My research focuses on leveraging RL techniques to develop intelligent control algorithms for various applications. I am fascinated by the potential of RL to enhance the capabilities of autonomous systems. Additionally I also hold strong experience in computer vision from object detection to computational geometry (3D reconstruction) through my senior design team UAV Forge, where I have held co-lead and lead responsibilities for the vision subteam.

We present a Proximal Policy Optimization (PPO) Reinforcement Learning algorithm for three-dimensional autonomous spacecraft trajectory planning. Specifically, we consider a chaser spacecraft performing a rendezvous and docking mission with a target spacecraft on a circular orbit. This reinforcement learning approach utilizes an actor and critic method to plan safe trajectories for the chaser spacecraft given constraints on its motion, including maximum thrust and line-of-sight constraints. We consider a fully actuated chaser spacecraft capable of applying continuous thrust in all three dimensions. Given this action space, we train a PPO model to perform rendezvous and docking maneuvers using spacecraft relative motion dynamics. We describe the training procedure and environment in detail and present results of nu- merous simulations showing that the trained model produces successful rendezvous and docking trajectories that satisfy line-of-sight constraints, even with significant random variations in initial conditions

Link to presentation at AAS Guidance, Navigation and Control (GN&C) Conference, 2023 [slides]

Link to publication [here]

Advisor Professor Copp [website]

First Author Vincent Beau Chen

Chen, V., Phillips, S. A., Copp, D. A., Planning Autonomous Spacecraft Rendezvous and Docking Trajectories via Reinforcement Learning Proceedings of the AAS Guidance, Navigation and Control (GN&C) Conference, 2023.

This work was supported by UCI’s Undergraduate Research Opportunities Program (UROP) and the Air Force Office of Scientific Research (AFOSR) through the Air Force Research Laboratory 16

100 Simulated Missions 1 Simulated Mission 100 Simulated Missions Conventional

Python3.10 Ubuntu20.04 Cuda-12.0 Vulkan-1.2 at the time of development.

Archetecture : x86_64

Simply clone the github and run the provided scripts. The scripts in benchmark are for recording metrics and visualization and training for transfer learning or new model training.