Wilderness Search and Rescue Missions using Deep Reinforcement Learning
Authors:
- 4 November 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 102-107
Abstract
Wilderness Search and Rescue (WiSAR) requires navigating large regions - often in rugged, remote areas - searching for missing people or animals. Because of the large regions and potentially limited mobility of ground vehicles, WiSAR missions are frequently carried out with the help of Unmanned Aerial Vehicles (UAVs). However, the ability to autonomously execute WiSAR remains an unsolved challenge. In this paper, we take advantage of Deep Reinforcement Learning (DRL) to develop an autonomous WiSAR controller for UAVs. We improve the learning and understanding of a UAV agent to explore a partially observable environment in search of a victim trapped in the wild. The proposed approach breaks up this difficult problem into 4 sub-tasks: tractable mapping of the environment in small regions, region selection, target search, and region exploration. Quad-Tree is utilized offline to decompose the environment map into smaller, tractable maps. Then, an efficient cost function is repeatedly computed to determine the best target region to search in each iteration of the process. Recurrent-DDQN and A2C algorithms are trained to generate optimal policies for the target search and regions exploration tasks, respectively. We tested our approach against a baseline of a hard-coded policy of navigating the map in a zigzag fashion and another baseline of using the same sub-tasks but instead of using the DRL algorithms, randomly selecting an action at each time step. The results demonstrate that our proposed approach is capable of navigating through 25 randomly generated environments and finding the missing victim faster than the baselines by 46%.Keywords
This publication has 9 references indexed in Scilit:
- Unsupervised Human Detection with an Embedded Vision System on a Fully Autonomous UAV for Search and Rescue OperationsSensors, 2019
- Autonomous Control of Unmanned Aerial VehiclesElectronics, 2019
- Task Allocation in Uncertain Environments using a QuadTree and Flow network2018
- Trajectory Optimization for Autonomous Flying Base Station via Reinforcement Learning2018
- Learning to fly by crashing2017
- Autonomous navigation of UAV by using real-time model-based reinforcement learning2017
- Receding Horizon "Next-Best-View" Planner for 3D Exploration2016
- DisCoF: Cooperative Pathfinding in Distributed Systems with Limited Sensing and Communication Range2016
- A Complete and Scalable Strategy for Coordinating Multiple Robots Within RoadmapsIEEE Transactions on Robotics, 2008