COLREGs-Based Path Planning for USVs Using the Deep Reinforcement Learning Strategy
Open Access
- 11 December 2023
- journal article
- research article
- Published by MDPI AG in Journal of Marine Science and Engineering
- Vol. 11 (12) , 2334
Abstract
This research introduces a two-stage deep reinforcement learning approach for the cooperative path planning of unmanned surface vehicles (USVs). The method is designed to address cooperative collision-avoidance path planning while adhering to the International Regulations for Preventing Collisions at Sea (COLREGs) and considering the collision-avoidance problem within the USV fleet and between USVs and target ships (TSs). To achieve this, the study presents a dual COLREGs-compliant action-selection strategy to effectively manage the vessel-avoidance problem. Firstly, we construct a COLREGs-compliant action-evaluation network that utilizes a deep learning network trained on pre-recorded TS avoidance trajectories by USVs in compliance with COLREGs. Then, the COLREGs-compliant reward-function-based action-selection network is proposed by considering various TS encountering scenarios. Consequently, the results of the two networks are fused to select actions for cooperative path-planning processes. The path-planning model is established using the multi-agent proximal policy optimization (MAPPO) method. The action space, observation space, and reward function are tailored for the policy network. Additionally, a TS detection method is introduced to detect the motion intentions of TSs. The study conducted Monte Carlo simulations to demonstrate the strong performance of the planning method. Furthermore, experiments focusing on COLREGs-based TS avoidance were carried out to validate the feasibility of the approach. The proposed TS detection model exhibited robust performance within the defined task.Keywords
Funding Information
- Natural Science Foundation of Liaoning Province (2020-MS-135, 62272067, MD-IPAC-2019103)
- National Natural Science Foundation of China (2020-MS-135, 62272067, MD-IPAC-2019103)
- Key Laboratory of Intelligent Perception and Advanced Control of State Ethnic Affairs Commission (2020-MS-135, 62272067, MD-IPAC-2019103)
This publication has 29 references indexed in Scilit:
- Intelligent collision avoidance algorithms for USVs via deep reinforcement learning under COLREGsOcean Engineering, 2020
- COLREG-Compliant Collision Avoidance for Unmanned Surface Vehicle Using Deep Reinforcement LearningIEEE Access, 2020
- Automatic ship collision avoidance using deep reinforcement learning with LSTM in continuous action spacesJournal of Marine Science and Technology, 2020
- Swarm control with collision avoidance for multiple underactuated surface vehiclesOcean Engineering, 2019
- A Novel Distributed and Self-Organized Swarm Control Framework for Underactuated Unmanned Marine VehiclesIEEE Access, 2019
- SeaShips: A Large-Scale Precisely Annotated Dataset for Ship DetectionIEEE Transactions on Multimedia, 2018
- A review on improving the autonomy of unmanned surface vehicles through intelligent collision avoidance manoeuvresAnnual Reviews in Control, 2012
- Collision risk assessment for shipsJournal of Marine Science and Technology, 2010
- Autonomous Ship Collision Avoidance Navigation Concepts, Technologies and TechniquesJournal of Navigation, 2007
- Obstacle avoidance in a dynamic environment: a collision cone approachIEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 1998