Hierarchical DDPG Based Reinforcement Learning Framework for Multi-Agent Collective Motion With Short Communication Ranges
Open Access
- 13 May 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Machine Learning in Communications and Networking
- Vol. 2, 633-644
Abstract
Collective motion is an important research content in the multi-agent control field. However, existing multi-agent collective motion methods typically assume large communication ranges of individual agents; in the scenario of leader-follower control with short communication ranges, if the leader dynamically changes its velocity without considering the followers’ states, the communication topology may be easily disconnected, making multi-agent collective motion more challenging. In this work, a novel Hierarchical DeepDeterministic PolicyGradient (HDDPG) based reinforcement learning framework is proposed to realize multi-agent collective motion with short communication ranges, ensuring the communication topology connected as much as possible. In H-DDPG, multiple agents with one single leader and numerous followers are dynamically divided into several hierarchies to conduct distributed control when the leader’s velocity changes. Two algorithms based on DDPG and the hierarchical strategy are designed to train followers in the first layer and followers in layers other than the first layer separately, which ensures that the agents form a tight swarm from scattered distribution and all followers can track the leader effectively. The experimental results demonstrate that with short communication ranges, H-DDPG outperforms the hierarchical flocking method in keeping the communication topology connection and shaping a tighter swarm.Keywords
Funding Information
- National Key Research and Development Program for young scientists of China (2022YFB3102800)
- Key Science and Technology Project of Henan Province: Research and Demonstration Application of Key Technology for Endogenous Safety in Intelligent Networked Vehicles (221100240100)
This publication has 27 references indexed in Scilit:
- Prescribed performance formation control for second-order multi-agent systems with connectivity and collision constraintsAutomatica, 2023
- A graph neural network based deep reinforcement learning algorithm for multi-agent leader-follower flockingInformation Sciences, 2023
- Graph attention mechanism based reinforcement learning for multi-agent flocking control in communication-restricted environmentInformation Sciences, 2022
- Multi-agent deep reinforcement learning: a surveyArtificial Intelligence Review, 2021
- Cooperative control for swarming systems based on reinforcement learning in unknown dynamic environmentNeurocomputing, 2020
- Robust finite-time connectivity preserving coordination of second-order multi-agent systemsAutomatica, 2018
- Erratum to: Why We Live in Hierarchies?2017
- A novel analysis on the efficiency of hierarchy among leader-following systemsAutomatica, 2016
- Collective motionPhysics Reports, 2012
- An Introduction to Multi-Agent Systems2010