Information-Bottleneck-Based Behavior Representation Learning for Multi-Agent Reinforcement Learning
Authors:
- 11 August 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-5
Abstract
In multi-agent deep reinforcement learning, extracting sufficient and compact information of other agents is critical to attain efficient convergence and scalability of an algorithm. In canonical frameworks, distilling of such information is often done in an implicit and uninterpretable manner, or explicitly with cost functions not able to reflect the relationship between information compression and utility in representation. In this paper, we present Information-Bottleneck-based Other agents’ behavior Representation learning for Multi-agent reinforcement learning (IBORM) to explicitly seek low-dimensional mapping encoder through which a compact and informative representation relevant to other agents’ behaviors is established. IBORM leverages the information bottleneck principle to compress observation information, while retaining sufficient information relevant to other agents’ behaviors used for cooperation decision. Empirical results have demonstrated that IBORM delivers the fastest convergence rate and the best performance of the learned policies, as compared with implicit behavior representation learning and explicit behavior representation learning without explicitly considering information compression and utility.Keywords
All Related Versions
This publication has 6 references indexed in Scilit:
- Stabilizing Multi-Agent Deep Reinforcement Learning by Implicitly Estimating Other Agents’ Behaviors2020
- Combined Reinforcement Learning via Abstract RepresentationsProceedings of the AAAI Conference on Artificial Intelligence, 2019
- Efficient Multi-agent Cooperative Navigation in Unknown Environments with Interlaced Deep Reinforcement Learning2019
- Deep learning and the information bottleneck principle2015
- Human-level control through deep reinforcement learningNature, 2015
- Deep auto-encoder neural networks in reinforcement learning2010