CCL: Collaborative Curriculum Learning for Sparse-Reward Multi-Agent Reinforcement Learning via Co-evolutionary Task Evolution
Preprint
- 9 May 2025
- preprint
- Published by MDPI AG in Preprints.org
Abstract
Sparse reward environments pose significant challenges in reinforcement learning, especially within multi-agent systems (MAS) where feedback is delayed and shared across agents, leading to suboptimal learning. We propose Collaborative Multi-dimensional Course Learning (CCL), a novel curriculum learning framework that addresses this by (1) refining intermediate tasks for individual agents, (2) using a variational evolutionary algorithm to generate informative subtasks, and (3) co-evolving agents with their environment to enhance training stability. Experiments on five cooperative tasks in the MPE and Hide-and-Seek environments show that CCL outperforms existing methods in sparse reward settings.Keywords
All Related Versions
This publication has 20 references indexed in Scilit:
- The Perils of Trial-and-Error Reward Design: Misdesign through Overfitting and Invalid Task SpecificationsProceedings of the AAAI Conference on Artificial Intelligence, 2023
- A model of symbiomemesis: machine education and communication as pillars for human-autonomy symbiosisPhilosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 2021
- Multi-agent reinforcement learning for redundant robot control in task-spaceInternational Journal of Machine Learning and Cybernetics, 2020
- A survey and critique of multiagent deep reinforcement learningAutonomous Agents and Multi-Agent Systems, 2019
- Self-Paced Prioritized Curriculum Learning With Coverage Penalty in Deep Reinforcement LearningIEEE Transactions on Neural Networks and Learning Systems, 2018
- Coevolutionary Multiobjective Evolutionary Algorithms: Survey of the State-of-the-ArtIEEE Transactions on Evolutionary Computation, 2017
- Evolution strategies – A comprehensive introductionNatural Computing, 2002
- Language acquisition in the absence of explicit negative evidence: how important is starting small?Cognition, 1999
- Catastrophic forgetting in connectionist networksTrends in Cognitive Sciences, 1999
- Learning and development in neural networks: the importance of starting smallCognition, 1993