A Survey and Critique of Multiagent Deep Reinforcement Learning
Preprint
Abstract
Deep reinforcement learning (RL) has achieved outstanding results in recent years. This has led to a dramatic increase in the number of applications and methods. Recent works have explored learning beyond single-agent scenarios and have considered multiagent learning (MAL) scenarios. Initial results report successes in complex multiagent domains, although there are several challenges to be addressed. The primary goal of this article is to provide a clear overview of current multiagent deep reinforcement learning (MDRL) literature. Additionally, we complement the overview with a broader analysis: (i) we revisit previous key components, originally presented in MAL and RL, and highlight how they have been adapted to multiagent deep reinforcement learning settings. (ii) We provide general guidelines to new practitioners in the area: describing lessons learned from MDRL works, pointing to recent benchmarks, and outlining open avenues of research. (iii) We take a more critical tone raising practical challenges of MDRL (e.g., implementation and computational demands). We expect this article will help unify and motivate future research to take advantage of the abundant literature that exists (e.g., RL and MAL) in a joint effort to promote fruitful research in the multiagent community.Keywords
All Related Versions
This publication has 84 references indexed in Scilit:
- Cooperative Multi-agent Policy Gradient2019
- Weighted Double Deep Multiagent Reinforcement Learning in Stochastic Cooperative Environments2018
- A Concise Introduction to Decentralized POMDPs2016
- Multiagent learning in the presence of memory-bounded agentsAutonomous Agents and Multi-Agent Systems, 2013
- Analyzing and visualizing multiagent rewards in dynamic and stochastic domainsAutonomous Agents and Multi-Agent Systems, 2008
- Cooperative Multi-Agent Learning: The State of the ArtAutonomous Agents and Multi-Agent Systems, 2005
- Learning Rates for Q-LearningLecture Notes in Computer Science, 2001
- Evolutionary computing in multi-agent environments: Operators1998
- Fictitious Play Property for Games with Identical InterestsJournal of Economic Theory, 1996
- Rule-injection hints as a means of improving network performance and learning time1990