MO-MIX: Multi-Objective Multi-Agent Cooperative Decision-Making With Deep Reinforcement Learning
- 7 June 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 45 (10) , 12098-12112
Abstract
Deep reinforcement learning (RL) has been applied extensively to solve complex decision-making problems. In many real-world scenarios, tasks often have several conflicting objectives and may require multiple agents to cooperate, which are the multi-objective multi-agent decision-making problems. However, only few works have been conducted on this intersection. Existing approaches are limited to separate fields and can only handle multi-agent decision-making with a single objective, or multi-objective decision-making with a single agent. In this paper, we propose MO-MIX to solve the multi-objective multi-agent reinforcement learning (MOMARL) problem. Our approach is based on the centralized training with decentralized execution (CTDE) framework. A weight vector representing preference over the objectives is fed into the decentralized agent network as a condition for local action-value function estimation, while a mixing network with parallel architecture is used to estimate the joint action-value function. In addition, an exploration guide approach is applied to improve the uniformity of the final non-dominated solutions. Experiments demonstrate that the proposed method can effectively solve the multi-objective multi-agent cooperative decision-making problem and generate an approximation of the Pareto set. Our approach not only significantly outperforms the baseline method in all four kinds of evaluation metrics, but also requires less computational cost.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62022094, 61988101)
- Zhejiang Lab (2021NB0AB01)
- Open Fund of the Laboratory of Cognition and Decision Intelligence for Complex Systems
- Chinese Academy of Sciences (CASIA-KFKT-03)
This publication has 33 references indexed in Scilit:
- Multiagent cooperation and competition with deep reinforcement learningPLOS ONE, 2017
- Human-level control through deep reinforcement learningNature, 2015
- Multiobjective Reinforcement Learning: A Comprehensive OverviewIEEE Transactions on Systems, Man, and Cybernetics: Systems, 2014
- A novel adaptive weight selection algorithm for multi-objective multi-agent reinforcement learning2014
- Adaptive multi-objective reinforcement learning with hybrid exploration for traffic signal control based on cooperative multi-agent frameworkEngineering Applications of Artificial Intelligence, 2014
- Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation2014
- MOEA/D: A Multiobjective Evolutionary Algorithm Based on DecompositionIEEE Transactions on Evolutionary Computation, 2007
- The Hypervolume Indicator Revisited: On the Design of Pareto-compliant Indicators Via Weighted Integration2007
- Multi-Agent Reinforcement Learning: Independent vs. Cooperative Agents1993
- Q-learningMachine Learning, 1992