Learning in Cooperative Multiagent Systems Using Cognitive and Machine Models
Open Access
- 14 October 2023
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Autonomous and Adaptive Systems
- Vol. 18 (4) , 1-22
Abstract
Developing effective multi-agent systems (MASs) is critical for many applications requiring collaboration and coordination with humans. Despite the rapid advance of multi-agent deep reinforcement learning (MADRL) in cooperative MASs, one of the major challenges that remain is the simultaneous learning and interaction of independent agents in dynamic environments in the presence of stochastic rewards. State-of-the-art MADRL models struggle to perform well in Coordinated Multi-agent Object Transportation Problems (CMOTPs) wherein agents must coordinate with each other and learn from stochastic rewards. In contrast, humans often learn rapidly to adapt to non-stationary environments that require coordination among people. In this article, motivated by the demonstrated ability of cognitive models based on Instance-based Learning Theory (IBLT) to capture human decisions in many dynamic decision-making tasks, we propose three variants of multi-agent IBL models (MAIBLs). The idea of these MAIBL algorithms is to combine the cognitive mechanisms of IBLT and the techniques of MADRL models to deal with coordination MASs in stochastic environments from the perspective of independent learners. We demonstrate that the MAIBL models exhibit faster learning and achieve better coordination in a dynamic CMOTP task with various settings of stochastic rewards compared to current MADRL models. We discuss the benefits of integrating cognitive insights into MADRL models.Keywords
All Related Versions
Funding Information
- Defense Advanced Research Projects Agency (W911NF-20-1-0006)
- AFRL award (FA8650-20-F-6212)
This publication has 29 references indexed in Scilit:
- Multi-agent reinforcement learning as a rehearsal for decentralized planningNeurocomputing, 2016
- Achieving Socially Optimal Outcomes in Multiagent Systems with Reinforcement Social LearningACM Transactions on Autonomous and Adaptive Systems, 2013
- Independent reinforcement learners in cooperative Markov games: a survey regarding coordination problemsThe Knowledge Engineering Review, 2012
- Instance-based learning: Integrating sampling and repeated decisions from experience.Psychological Review, 2011
- Instance‐based Learning: A General Model of Repeated Binary ChoiceJournal of Behavioral Decision Making, 2010
- A multiagent approach to managing air traffic flowAutonomous Agents and Multi-Agent Systems, 2010
- Cooperation through self-assembly in multi-robot systemsACM Transactions on Autonomous and Adaptive Systems, 2006
- Exploring selfish reinforcement learning in repeated games with stochastic rewardsAutonomous Agents and Multi-Agent Systems, 2006
- Instance‐based learning in dynamic decision makingCognitive Science, 2003
- Q-learningMachine Learning, 1992