A Coordination Optimization Framework for Multi-Agent Reinforcement Learning Based on Reward Redistribution and Experience Reutilization
Open Access
- 9 June 2025
- journal article
- research article
- Published by MDPI AG in Electronics
- Vol. 14 (12) , 2361
Abstract
Cooperative multi-agent reinforcement learning (MARL) has emerged as a powerful paradigm for addressing complex real-world challenges, including autonomous robot control, strategic decision-making, and decentralized coordination in unmanned swarm systems. However, it still faces challenges in learning proper coordination among multiple agents. The lack of effective knowledge sharing and experience interaction mechanisms among agents has led to substantial performance decline, especially in terms of low sampling efficiency and slow convergence rates, ultimately constraining the practical applicability of MARL. To address these challenges, this paper proposes a novel framework termed Reward redistribution and Experience reutilization based Coordination Optimization (RECO). This innovative approach employs a hierarchical experience pool mechanism that enhances exploration through strategic reward redistribution and experience reutilization. The RECO framework incorporates a sophisticated evaluation mechanism that assesses the quality of historical sampling data from individual agents and optimizes reward distribution by maximizing mutual information across hierarchical experience trajectories. Extensive comparative analyses of computational efficiency and performance metrics across diverse environments reveal that the proposed method not only enhances training efficiency in multi-agent gaming scenarios but also significantly strengthens algorithmic robustness and stability in dynamic environments.Keywords
Funding Information
- National Key Laboratory of Land and Air Based Information Perception and Control, China (B124003, 62388101, 2024JC-YBMS-560)
- Anhui Province Key Laboratory of Machine Vision Detection and Perception (B124003, 62388101, 2024JC-YBMS-560)
- Basic Science Center Program of the National Natural Science Foundation of China (B124003, 62388101, 2024JC-YBMS-560)
- Shaanxi Province Natural Science Basic Research Program (B124003, 62388101, 2024JC-YBMS-560)
This publication has 21 references indexed in Scilit:
- A Resource-Aware Multi-Graph Neural Network for Urban Traffic Flow Prediction in Multi-Access Edge Computing SystemsIEEE Transactions on Consumer Electronics, 2024
- Prioritized experience replay based on dynamics priorityScientific Reports, 2024
- Re-attentive experience replay in off-policy reinforcement learningMachine Learning, 2024
- Guided probabilistic reinforcement learning for sampling-efficient maintenance scheduling of multi-component systemApplied Mathematical Modelling, 2023
- Cooperative multi-agent target searching: a deep reinforcement learning approach based on parallel hindsight experience replayComplex & Intelligent Systems, 2023
- Group-Based Distributed Auction Algorithms for Multi-Robot Task AssignmentIEEE Transactions on Automation Science and Engineering, 2022
- Discovering diverse solutions in deep reinforcement learning by maximizing state–action-based mutual informationNeural Networks, 2022
- Lucid dreaming for experience replay: refreshing past states with the current policyNeural Computing and Applications, 2021
- Integrated Task Assignment and Path Planning for Capacitated Multi-Agent Pickup and DeliveryIEEE Robotics and Automation Letters, 2021
- Grandmaster level in StarCraft II using multi-agent reinforcement learningNature, 2019