Attentive Relational State Representation in Decentralized Multiagent Reinforcement Learning

Authors:
Publisher website
Google Scholar
Abstract
In multiagent reinforcement learning (MARL), it is crucial for each agent to model the relation with its neighbors. Existing approaches usually resort to concatenate the features of multiple neighbors, fixing the size and the identity of the inputs. But these settings are inflexible and unscalable. In this article, we propose an attentive relational encoder (ARE), which is a novel scalable feedforward neural module, to attentionally aggregate an arbitrary-sized neighboring feature set for state representation in the decentralized MARL. The ARE actively selects the relevant information from the neighboring agents and is permutation invariant, computationally efficient, and flexible to interactive multiagent systems. Our method consistently outperforms the latest competing decentralized MARL methods in several multiagent tasks. In particular, it shows strong cooperative performance in challenging StarCraft micromanagement tasks and achieves over a 96% winning rate against the most difficult noncheating built-in artificial intelligence bots.
Funding Information
  • Science and Technology Innovation 2030—“New Generation Artificial Intelligence” Major Project (2018AAA0102301, 2018AAA0100302)
  • National Natural Science Foundation of China (61673025, 61375119)
  • National Key Basic Research Development Plan (973 Plan) Project of China (2015CB352302)

This publication has 23 references indexed in Scilit: