Abstract
This article investigates the application of deep reinforcement learning (RL) in the reactive collision avoidance control of an autonomous underwater vehicle (AUV) using multibeam forward-looking sonar (MBFLS). First, an efficient encoding method based on prior knowledge is proposed, which can extract low dimensional representations of obstacles from massive MBFLS detection data, thereby reducing the inference time of the reactive collision avoidance control policy represented by a neural network and improving their collision avoidance effect. On this basis, a state space, action space, and multiobjective reward function are customized for the reactive collision avoidance task of the AUV, and a simulation training environment is constructed. Considering that the contradiction between policy training and deployment is caused by unknown total disturbances, disturbance estimators based on the extended state observer is constructed, and a deployment scheme is designed to compensate for disturbances through both feedforward and feedback channels, ensuring that the policy learned in simulation can be applied to an actual AUV. Finally, combining the above components, a novel reactive collision avoidance method based on deep RL is proposed, and the effectiveness of the proposed method for obstacles of various shapes (both static and dynamic) is fully verified in three different unknown environments.
Funding Information
  • National Natural Science Foundation of China (U2141234)
  • National Science and Technology Major Project (2022ZD0119900)
  • Hainan Province Science and Technology Special Fund (ZDYF2024GXJS003)
  • Shanghai Science and Technology Program (22015810300)