Reinforcement Learning Boat Autopilot: A Sample-efficient and Model Predictive Control based Approach
Authors:
- 1 November 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2868-2875
Abstract
In this research we focus on developing a reinforcement learning system for a challenging task: autonomous control of a real-sized boat, with difficulties arising from large uncertainties in the challenging ocean environment and the extremely high cost of exploring and sampling with a real boat. To this end, we explore a novel Gaussian processes (GP) based reinforcement learning approach that combines sample-efficient model-based reinforcement learning and model predictive control (MPC). Our approach, sample-efficient probabilistic model predictive control (SPMPC), iteratively learns a Gaussian process dynamics model and uses it to efficiently update control signals within the MPC closed control loop. A system using SPMPC is built to efficiently learn an autopilot task. After investigating its performance in a simulation modeled upon real boat driving data, the proposed system successfully learns to drive a real-sized boat equipped with a single engine and sensors measuring GPS, speed, direction, and wind in an autopilot task without human demonstration.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- Unmanned surface vehicles: An overview of developments and challengesAnnual Reviews in Control, 2016
- Preparing a nation for autonomous vehicles: opportunities, barriers and policy recommendationsTransportation Research Part A: Policy and Practice, 2015
- Robust Adaptive Control of an Uninhabited Surface VehicleJournal of Intelligent & Robotic Systems, 2014
- Reinforcement learning in robotics: A surveyThe International Journal of Robotics Research, 2013
- COLREGs-based collision avoidance strategies for unmanned surface vehiclesMechatronics, 2012
- Improving autonomy and control of autonomous surface vehicles in port protection and mine countermeasure scenariosJournal of Field Robotics, 2010
- Sequential Quadratic Programming2006
- Tracking control of an underactuated shipIEEE Transactions on Control Systems Technology, 2003
- Constrained model predictive control: Stability and optimalityAutomatica, 2000
- Reinforcement Learning: An IntroductionIEEE Transactions on Neural Networks, 1998