Sim-to-Real Deep Reinforcement Learning for Safe End-to-End Planning of Aerial Robots
Open Access
Abstract
In this study, a novel end-to-end path planning algorithm based on deep reinforcement learning is proposed for aerial robots deployed in dense environments. The learning agent finds an obstacle-free way around the provided rough, global path by only depending on the observations from a forward-facing depth camera. A novel deep reinforcement learning framework is proposed to train the end-to-end policy with the capability of safely avoiding obstacles. The Webots open-source robot simulator is utilized for training the policy, introducing highly randomized environmental configurations for better generalization. The training is performed without dynamics calculations through randomized position updates to minimize the amount of data processed. The trained policy is first comprehensively evaluated in simulations involving physical dynamics and software-in-the-loop flight control. The proposed method is proven to have a 38% and 50% higher success rate compared to both deep reinforcement learning-based and artificial potential field-based baselines, respectively. The generalization capability of the method is verified in simulation-to-real transfer without further training. Real-time experiments are conducted with several trials in two different scenarios, showing a 50% higher success rate of the proposed method compared to the deep reinforcement learning-based baseline.Keywords
Funding Information
- the European Union’s Horizon 2020 Research and Innovation Program (OpenDR) (871449)
This publication has 34 references indexed in Scilit:
- Reinforcement Learning for UAV Attitude ControlACM Transactions on Cyber-Physical Systems, 2019
- Learning motion primitives for planning swift maneuvers of quadrotorAutonomous Robots, 2019
- An End-to-End Deep Reinforcement Learning-Based Intelligent Agent Capable of Autonomous Exploration in Unknown EnvironmentsSensors, 2018
- Agile Coordination and Assistive Collision Avoidance for Quadrotor Swarms Using Virtual StructuresIEEE Transactions on Robotics, 2018
- In Defense of Classical Image Processing: Fast Depth Completion on the CPU2018
- Differential Flatness of Quadrotor Dynamics Subject to Rotor Drag for Accurate Tracking of High-Speed TrajectoriesIEEE Robotics and Automation Letters, 2017
- Human-level control through deep reinforcement learningNature, 2015
- PIXHAWK: A system for autonomous flight using onboard computer vision2011
- Cyberbotics Ltd. Webots ™ : Professional Mobile Robot SimulationInternational Journal of Advanced Robotic Systems, 2004
- A Formal Basis for the Heuristic Determination of Minimum Cost PathsIEEE Transactions on Systems Science and Cybernetics, 1968