Revealing principles of autonomous thermal soaring in windy conditions using vulture-inspired deep reinforcement-learning
Preprint
Abstract
Thermal soaring, a technique used by birds and gliders to utilize updrafts of hot air, is an appealing model-problem for studying motion control and how it is learned by animals and engineered autonomous systems. Thermal soaring has rich dynamics and nontrivial constraints, yet it uses few control parameters and is becoming experimentally accessible. Following recent developments in applying reinforcement learning methods for training deep neural-network (deep-RL) models to soar autonomously both in simulation and real gliders, here we develop a simulation-based deep-RL system to study the learning process of thermal soaring. We find that this process has learning bottlenecks, we define a new efficiency metric and use it to characterize learning robustness, we compare the learned policy to data from soaring vultures, and find that the neurons of the trained network divide into function clusters that evolve during learning. These results pose thermal soaring as a rich yet tractable model-problem for the learning of motion control.Keywords
All Related Versions
- Published version: Nature Communications, 15 (1), 1.
This publication has 38 references indexed in Scilit:
- Reinforcement Learning Enables Resource Partitioning in Foraging BatsCurrent Biology, 2020
- Toward Interpretable Deep Reinforcement Learning with Linear Model U-Trees2019
- Glider soaring via reinforcement learning in the fieldNature, 2018
- The AutoSOAR autonomous soaring aircraft, part 1: Autonomy algorithmsJournal of Field Robotics, 2018
- Mastering the game of Go without human knowledgeNature, 2017
- The AutoSOAR autonomous soaring aircraft part 2: Hardware implementation and flight resultsJournal of Field Robotics, 2017
- Human-level control through deep reinforcement learningNature, 2015
- Atmospheric flow field models applicable for aircraft endurance extensionProgress in Aerospace Sciences, 2013
- Comparing bird and human soaring strategiesProceedings of the National Academy of Sciences, 2008
- The role of thermals in the convective boundary layerBoundary-Layer Meteorology, 1980