HMAAC: Hierarchical Multi-Agent Actor-Critic for Aerial Search with Explicit Coordination Modeling
Authors:
- 29 May 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 7728-7734
Abstract
Unmanned Aerial Vehicles (UAVs) have become prevalent in Search-And-Rescue (SAR) missions. However, existing solutions to the control and coordination of UAV s are mostly limited to specific environments and are not robust to handle unreliable/unstable communications. To deal with these challenges, Hierarchical Multi-Agent Actor-Critic (HMAAC) framework is proposed where a high-level policy is placed on top of individual low-level actor-critic policies to relax the inter-dependency among the agents. The low-level policies are considered conditionally independent given the coordination action, which is generated by the high-level policy. A Central-ized Training Decentralized Execution (CTDE) would not work because it cannot be assumed that communication is always perfect during training and that the whole system can rely on stable communications during deployment. The proposed framework is evaluated in AirSim, a realistic multi-UAV simula-tor, and is compared against two existing algorithms, i.e., Multi- Agent Actor-Critic (MAAC) and decentralized REINFORCE, in two scenarios, (a) when packet drop is modeled as a Bernoulli process and (b) when shadow zones are created in the search space and communication will be lost if the agents are in these zones. Results show that HMAAC is scalable and robust to unreliable communication and outperforms the other algorithms in terms of exploration and coordination when the number of agents is large and communications are not stable.Keywords
This publication has 16 references indexed in Scilit:
- Deep Multi-Task Learning for Anomalous Driving Detection Using CAN Bus Scalar Sensor Data2019
- Multi-agent Hierarchical Reinforcement Learning with Dynamic Termination2019
- Unsupervised Human Detection with an Embedded Vision System on a Fully Autonomous UAV for Search and Rescue OperationsSensors, 2019
- Robot Swarm Navigation and Victim Detection Using Rendezvous Consensus in Search and Rescue OperationsApplied Sciences, 2019
- LSAR: Multi-UAV Collaboration for Search and Rescue MissionsIEEE Access, 2019
- A Fully-Autonomous Aerial Robot for Search and Rescue Applications in Indoor Environments using Learning-Based TechniquesJournal of Intelligent & Robotic Systems, 2018
- AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles2017
- Active Autonomous Aerial Exploration for Ground Robot Path PlanningIEEE Robotics and Automation Letters, 2017
- Exploration strategies based on multi-criteria decision making for searching environments in rescue operationsAutonomous Robots, 2011
- Simple statistical gradient-following algorithms for connectionist reinforcement learningMachine Learning, 1992