Reinforcement learning based autonomous multi-rotor landing on moving platforms
Open Access
- 6 June 2024
- journal article
- research article
- Published by Springer Nature in Autonomous Robots
- Vol. 48 (4-5) , 1-21
Abstract
Multi-rotor UAVs suffer from a restricted range and flight duration due to limited battery capacity. Autonomous landing on a 2D moving platform offers the possibility to replenish batteries and offload data, thus increasing the utility of the vehicle. Classical approaches rely on accurate, complex and difficult-to-derive models of the vehicle and the environment. Reinforcement learning (RL) provides an attractive alternative due to its ability to learn a suitable control policy exclusively from data during a training procedure. However, current methods require several hours to train, have limited success rates and depend on hyperparameters that need to be tuned by trial-and-error. We address all these issues in this work. First, we decompose the landing procedure into a sequence of simpler, but similar learning tasks. This is enabled by applying two instances of the same RL based controller trained for 1D motion for controlling the multi-rotor’s movement in both the longitudinal and the lateral directions. Second, we introduce a powerful state space discretization technique that is based on i) kinematic modeling of the moving platform to derive information about the state space topology and ii) structuring the training as a sequential curriculum using transfer learning. Third, we leverage the kinematics model of the moving platform to also derive interpretable hyperparameters for the training process that ensure sufficient maneuverability of the multi-rotor vehicle. The training is performed using the tabular RL method Double Q-Learning. Through extensive simulations we show that the presented method significantly increases the rate of successful landings, while requiring less training time compared to other deep RL approaches. Furthermore, for two comparison scenarios it achieves comparable performance than a cascaded PI controller. Finally, we deploy and demonstrate our algorithm on real hardware. For all evaluation scenarios we provide statistics on the agent’s performance. Source code is openly available at https://github.com/robot-perception-group/rl_multi_rotor_landing.Keywords
All Related Versions
Funding Information
- Universität Stuttgart
This publication has 23 references indexed in Scilit:
- Autonomous Vehicular Landings on the Deck of an Unmanned Surface Vehicle using Deep Reinforcement LearningRobotica, 2019
- A Deep Reinforcement Learning Strategy for UAV Autonomous Landing on a Moving PlatformJournal of Intelligent & Robotic Systems, 2018
- Nonlinear and Adaptive Intelligent Control Techniques for Quadrotor UAV – A SurveyAsian Journal of Control, 2018
- Autonomous Landing of a Multirotor Micro Air Vehicle on a High Velocity Ground VehicleIFAC-PapersOnLine, 2017
- Vision Based Autonomous Landing of Multirotor UAV on Moving PlatformJournal of Intelligent & Robotic Systems, 2016
- RotorS—A Modular Gazebo MAV Simulator Framework2016
- Fast, safe and precise landing of a quadrotor on an oscillating platform2015
- Autonomous Maritime Landings for Low-Cost VTOL Aerial Vehicles2014
- Automatic Take Off, Tracking and Landing of a Miniature UAV on a Moving Carrier VehicleJournal of Intelligent & Robotic Systems, 2010
- Multiresolution state-space discretization method for Q-Learning2009