Deep-Reinforcement-Learning-Based Reactive Collision Avoidance Controller for an Underactuated AUV Using Multibeam Forward-Looking Sonar
- 25 June 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 12 (17) , 36604-36616
Abstract
This article investigates the application of deep reinforcement learning (RL) in the reactive collision avoidance control of an autonomous underwater vehicle (AUV) using multibeam forward-looking sonar (MBFLS). First, an efficient encoding method based on prior knowledge is proposed, which can extract low dimensional representations of obstacles from massive MBFLS detection data, thereby reducing the inference time of the reactive collision avoidance control policy represented by a neural network and improving their collision avoidance effect. On this basis, a state space, action space, and multiobjective reward function are customized for the reactive collision avoidance task of the AUV, and a simulation training environment is constructed. Considering that the contradiction between policy training and deployment is caused by unknown total disturbances, disturbance estimators based on the extended state observer is constructed, and a deployment scheme is designed to compensate for disturbances through both feedforward and feedback channels, ensuring that the policy learned in simulation can be applied to an actual AUV. Finally, combining the above components, a novel reactive collision avoidance method based on deep RL is proposed, and the effectiveness of the proposed method for obstacles of various shapes (both static and dynamic) is fully verified in three different unknown environments.Keywords
Funding Information
- National Natural Science Foundation of China (U2141234)
- National Science and Technology Major Project (2022ZD0119900)
- Hainan Province Science and Technology Special Fund (ZDYF2024GXJS003)
- Shanghai Science and Technology Program (22015810300)
This publication has 29 references indexed in Scilit:
- Improving Deep Q Network Based on Marketing Psychology for AUV Path Planning in Unknown Marine EnvironmentsIEEE Internet of Things Journal, 2024
- Information-Entropy-Based Trajectory Planning for AUV-Aided Network Localization: A Reinforcement Learning ApproachIEEE Internet of Things Journal, 2024
- AUV path tracking with real-time obstacle avoidance via reinforcement learning under adaptive constraintsOcean Engineering, 2022
- Magnetic control of tokamak plasmas through deep reinforcement learningNature, 2022
- Path planning and obstacle avoidance for AUV: A reviewOcean Engineering, 2021
- Mastering Atari, Go, chess and shogi by planning with a learned modelNature, 2020
- Closed-loop randomized kinodynamic path planning for an autonomous underwater vehicleApplied Ocean Research, 2019
- Human-level control through deep reinforcement learningNature, 2015
- Handbook of Marine Craft Hydrodynamics and Motion Control2011
- Analysis and design for the second order nonlinear continuous extended states observerChinese Science Bulletin, 2000