Reinforcement Learning-Driven Autonomous Path Planning for Unmanned Surface Vehicles: Current Status, Challenges, and Future Prospects
Open Access
Abstract
The continuous advancement of autonomy and intelligence in marine shipping has made the safe and efficient navigation of unmanned surface vehicles in complex waters a major research focus. As a key link of the autonomous decision-making system for unmanned surface vehicles (USVs), local path planning needs to achieve real-time collision avoidance and motion optimization under dynamic obstacles, multiple rule constraints, and strong environmental uncertainty. In recent years, reinforcement learning has gradually become an important technical route for local path planning of USVs by virtue of its autonomous decision-making ability in high-dimensional continuous state space and adaptability to complex nonlinear problems. Combined with the evolution of the algorithm paradigm and its functional positioning in different water scenarios, this paper systematically reviews the relevant literature by examining the evolution of algorithmic paradigms; focuses on summarizing deep Q-network (DQN), Proximal Policy Optimization (PPO), Soft Actor-Critic (SAC), and Twin Delayed Deep Deterministic Policy Gradient (TD3), along with the collaborative architectures integrated with traditional planning methods such as A* and Rapidly-exploring Random Tree (RRT); and summarizes the performance characteristics, advantages, and limitations of various methods in typical scenarios. The review shows that the main bottlenecks of current research include insufficient reward mechanism design, low sample utilization efficiency, difficulty in transferring from simulation to real ships, and insufficient safety and trustworthiness verification. This paper looks forward to the future development trends from the two directions of data fusion and security enhancement in order to provide reference for related research.Keywords
Funding Information
- Department of Education of Hebei Province (BJ2026303)
- Hebei University of Science and Technology (2023XLQ001)
This publication has 44 references indexed in Scilit:
- Path planning of indoor unmanned vehicle based on adaptive A* algorithmEngineering Applications of Artificial Intelligence, 2026
- Integrating renewable energy into automated ports through electric AGV: A reinforcement learning-based multi-tasks coordination method for optimal AGVs charging/loading strategyJournal of Energy Storage, 2026
- A multi-objective deep reinforcement learning framework for energy efficiency of autonomous harbor craftsApplied Energy, 2025
- Deep reinforcement learning for collision avoidance in unmanned surface vehicles: State-of-the-artApplied Ocean Research, 2025
- Reinforcement learning approaches in the motion systems of autonomous underwater vehiclesApplied Ocean Research, 2025
- USV formation navigation decision-making through hybrid deep reinforcement learning using self-attention mechanismExpert Systems with Applications, 2024
- A survey on large language model based autonomous agentsFrontiers of Computer Science, 2024
- Risk-based Convolutional Perception Models for Collision Avoidance in Autonomous Marine Surface Vessels using Deep Reinforcement LearningIFAC-PapersOnLine, 2023
- A path planning strategy unified with a COLREGS collision avoidance function based on deep reinforcement learning and artificial potential fieldApplied Ocean Research, 2021
- Local path optimization method for unmanned ship based on particle swarm acceleration calculation and dynamic optimal controlApplied Ocean Research, 2021