Investigating the Impact of Communication-Induced Action Space on Exploration of Unknown Environments with Decentralized Multi-Agent Reinforcement Learning
Preprint
Abstract
This paper introduces a novel enhancement to the Decentralized Multi-Agent Reinforcement Learning (D-MARL) exploration by proposing communication-induced action space to improve the mapping efficiency of unknown environments using homogeneous agents. Efficient exploration of large environments relies heavily on inter-agent communication as real-world scenarios are often constrained by data transmission limits, such as signal latency and bandwidth. Our proposed method optimizes each agent's policy using the heterogeneous-agent proximal policy optimization algorithm, allowing agents to autonomously decide whether to communicate or to explore, that is whether to share the locally collected maps or continue the exploration. We propose and compare multiple novel reward functions that integrate inter-agent communication and exploration, enhance mapping efficiency and robustness, and minimize exploration overlap. This article presents a framework developed in ROS2 to evaluate and validate the investigated architecture. Specifically, four TurtleBot3 Burgers have been deployed in a Gazebo-designed environment filled with obstacles to evaluate the efficacy of the trained policies in mapping the exploration arena.Keywords
All Related Versions
This publication has 14 references indexed in Scilit:
- Robot Operating System 2: Design, architecture, and uses in the wildScience Robotics, 2022
- Explore-Bench: Data Sets, Metrics and Evaluations for Frontier-based and Deep-reinforcement-learning-based Autonomous Exploration2022
- H2GNN: Hierarchical-Hops Graph Neural Networks for Multi-Robot Exploration in Unknown EnvironmentsIEEE Robotics and Automation Letters, 2022
- Multi-Robot Coverage and Exploration using Spatial Graph Neural Networks2021
- Message-Aware Graph Attention Networks for Large-Scale Multi-Robot Path PlanningIEEE Robotics and Automation Letters, 2021
- A Queue-Stabilizing Framework for Networked Multi-Robot ExplorationIEEE Robotics and Automation Letters, 2021
- Voronoi-Based Multi-Robot Autonomous Exploration in Unknown Environments via Deep Reinforcement LearningIEEE Transactions on Vehicular Technology, 2020
- Decentralized Strategy for Cooperative Multi-Robot Exploration and MappingIFAC-PapersOnLine, 2020
- Deep Reinforcement Learning Robot for Search and Rescue Applications: Exploration in Unknown Cluttered EnvironmentsIEEE Robotics and Automation Letters, 2019
- Multirobot Exploration of Communication-Restricted Environments: A SurveyIEEE Intelligent Systems, 2017