Autonomous Reinforcement Control of Visual Underwater Vehicles: Real-Time Experiments Using Computer Vision
- 24 May 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Vehicular Technology
- Vol. 71 (8) , 8237-8250
Abstract
Swift decision-making based on visual environment perception is crucial for autonomous control of visual underwater vehicles (VUVs) during underwater missions. However, learning perception and decision models individually might result in weak robustness of overall control system as the mismatched state extraction and control decision making are asynchronous. As a remedy, we will introduce in this paper an end-to-end monocular autonomous reinforcement control (MARC) framework for autonomous control of VUVs, which is performed in two cascaded procedures, i.e., 1) perception, where a geometric network (GeoNet) is designed based on a convolutional encoder-decoder network to generate depth maps from input environmental videos; 2) decision, where with depth maps as input, a reinforcement control network (CtrlNet) integrates a convolutional neural network into a deep deterministic policy gradient network and outputs action decisions, which are refined by reinforcement learning algorithm for obstacle-avoiding based autonomous control. Numerical and experimental results demonstrate that the proposed MARC exhibits high-quality depth prediction and is capable of conducting obstacle-avoiding navigation and autonomous control of VUVs with high accuracy and strong robustness.Keywords
Funding Information
- National Natural Science Foundation of China (51979021, 51709028)
- Natural Science Foundation of Liaoning (2019JH8/10100045)
- Dalian High-level Talent Innovation Support Program (2019RQ008)
- Fundamental Research Funds for the Central Universities (3132022218, 3132019317)
This publication has 31 references indexed in Scilit:
- Fully-tuned fuzzy neural network based robust adaptive tracking control of unmanned underwater vehicle with thruster dynamicsNeurocomputing, 2016
- A robust dynamic region-based control scheme for an autonomous underwater vehicleOcean Engineering, 2016
- A Machine Learning Approach to Visual Perception of Forest Trails for Mobile RobotsIEEE Robotics and Automation Letters, 2015
- Human-level control through deep reinforcement learningNature, 2015
- Disturbance observer based tracking controller for an autonomous underwater vehicle2015
- The Ornstein–Uhlenbeck process as a model of a low pass filtered white noiseMetrologia, 2008
- Trajectory Planning and Tracking Control of Underactuated AUVs2005
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004
- A parallel stereo algorithm that produces dense depth maps and preserves image featuresMachine Vision and Applications, 1993
- The Monte Carlo MethodJournal of the American Statistical Association, 1949