Enhancing Multi-Agent Cooperation Through Action-Probability-Based Communication
Open Access
- 20 June 2024
- journal article
- research article
- Published by Fuji Technology Press Ltd. in Journal of Robotics and Mechatronics
- Vol. 36 (3) , 658-668
Abstract
Although communication plays a pivotal role in achieving coordinated activities in multi-agent systems, conventional approaches often involve complicated high-dimensional messages generated by deep networks. These messages are typically indecipherable to humans, are relatively costly to transmit, and require intricate encoding and decoding networks. This can pose a design limitation for the agents such as autonomous (mobile) robots. This lack of interpretability can lead to systemic issues with security and reliability. In this study, inspired by common human communication about likely actions in collaborative endeavors, we propose a novel approach in which each agent’s action probabilities are transmitted to other agents as messages, drawing inspiration from the common human practice of sharing likely actions in collaborative endeavors. Our proposed framework is referred to as communication based on action probabilities (CAP), and focuses on generating straightforward, low-dimensional, interpretable messages to support multiple agents in coordinating their activities to achieve specified cooperative goals. CAP streamlines our comprehension of the agents’ learned coordinated and cooperative behaviors and eliminates the need to use additional network models to generate messages. CAP’s network architecture is simpler than that of state-of-the-art methods, and our experimental results show that it nonetheless performed comparably, converged faster, and exhibited a lower volume of communication with better interpretability.Keywords
Funding Information
- Japan Science and Technology Agency (20H04245)
- SPRING (JPMJSP2128)
This publication has 6 references indexed in Scilit:
- Learning to Communicate Using Action Probabilities for Multi-Agent Cooperation2023
- Composing Synergistic Macro Actions for Reinforcement Learning AgentsIEEE Transactions on Neural Networks and Learning Systems, 2022
- Learning Agent Communication under Limited Bandwidth by Message PruningProceedings of the AAAI Conference on Artificial Intelligence, 2020
- Sharing Experience for Behavior Generation of Real Swarm Robot Systems Using Deep Reinforcement LearningJournal of Robotics and Mechatronics, 2019
- Children’s strategic theory of mindProceedings of the National Academy of Sciences, 2014
- Multi-Agent Reinforcement Learning: Independent vs. Cooperative Agents1993