Spacecraft Safe Proximity Policy Based on Graph Neural Network Safe Reinforcement Learning
Open Access
Abstract
Spacecraft safe proximity, as a critical component of on-orbit servicing missions, primarily encounters the following two challenges: the partial observability of the environment surrounding the service spacecraft and the necessity to evade uncertain obstacles. A safe reinforcement learning algorithm based on a graph neural network is proposed to address the constrained Markov decision problem in partially observable scenarios for spacecraft safe proximity missions. A graph neural network mechanism is introduced to solve the problem of dynamic variations in the quantity and location of obstacles in the observation area of the service spacecraft. The graph attention network is used to facilitate the extraction of feature information from the graph structure, which is then utilized as input for the subsequent reinforcement learning algorithm. The Soft Actor–Critic–Lagrangian algorithm is adopted to deal with the problems of tuning reward function parameters and balancing safety and optimality. By introducing Lagrange multipliers, the constrained optimization problem is transformed into an unconstrained optimization problem. In order to verify the effectiveness of the algorithm proposed in this paper, a spacecraft safe proximity environment model with dynamic obstacles is constructed, and the GAT-SACL algorithm proposed in this paper is validated by the Monte Carlo shooting method. The results show that the GAT-SACL algorithm possess excellent exploratory characteristics and delivers significant advantages in balancing optimality and safety.Keywords
Funding Information
- National Natural Science Foundation of China (12502410)
- National Natural Science Foundation of China (12472047)
- National Natural Science Foundation of China (62401597)
- the Foundation of the National Key Laboratory of Science and Technology on Space-Born Intelligent Information Processing (TJ-03-25-01)
- Natural Science Foundation of Hainan Province (2025JJ50016)
This publication has 31 references indexed in Scilit:
- Graph-based multi-agent reinforcement learning for large-scale UAVs swarm system controlAerospace Science and Technology, 2024
- Autonomous spacecraft collision avoidance with a variable number of space debris based on safe reinforcement learningAerospace Science and Technology, 2024
- Meta-reinforcement learning for adaptive spacecraft guidance during finite-thrust rendezvous missionsActa Astronautica, 2022
- Reinforcement learning in spacecraft control applications: Advances, prospects, and challengesAnnual Reviews in Control, 2022
- Relative motion guidance for near-rectilinear lunar orbits with path constraints via actor-critic reinforcement learningAdvances In Space Research, 2022
- Near-optimal interception strategy for orbital pursuit-evasion using deep reinforcement learningActa Astronautica, 2022
- Connectivity preservation and collision avoidance control for spacecraft formation flying in the presence of multiple obstaclesAdvances In Space Research, 2021
- Graph neural networks: A review of methods and applicationsAI Open, 2020
- Spacecraft formation reconfiguration with multi-obstacle avoidance under navigation and control uncertainties using adaptive artificial potential function methodAstrodynamics, 2019
- Safe-trajectory optimization and tracking control in ultra-close proximity to a failed satelliteActa Astronautica, 2018