A Distributed Multi-Agent Formation Control Method Based on Deep Q Learning
Authors:
Open Access
- 31 March 2022
- journal article
- research article
- Published by Frontiers Media SA in Frontiers in Neurorobotics
- Vol. 16, 817168
Abstract
Distributed control method plays an important role in the formation of a multi-agent system (MAS), which is the prerequisite for an MAS to complete its missions. However, the lack of considering the collision risk between agents makes many distributed formation control methods lose practicability. In this article, a distributed formation control method that takes collision avoidance into account is proposed. At first, the MAS formation control problem can be divided into pair-wise unit formation problems where each agent moves to the expected position and only needs to avoid one obstacle. Then, a deep Q network (DQN) is applied to model the agent's unit controller for this pair-wise unit formation. The DQN controller is trained by using reshaped reward function and prioritized experience replay. The agents in MAS formation share the same unit DQN controller but get different commands due to various observations. Finally, through the min-max fusion of value functions of the DQN controller, the agent can always respond to the most dangerous avoidance. In this way, we get an easy-to-train multi-agent collision avoidance formation control method. In the end, unit formation simulation and multi-agent formation simulation results are presented to verify our method.Keywords
This publication has 16 references indexed in Scilit:
- Decentralized behavior-based formation control of multiple robots considering obstacle avoidanceIntelligent Service Robotics, 2017
- Collision avoidance for mobile robots based on artificial potential field and obstacle envelope modellingAssembly Automation, 2016
- Implementation Studies of Robot Swarm Navigation Using Potential Functions and Panel MethodsIEEE/ASME Transactions on Mechatronics, 2016
- Review: Milking robot utilization, a successful precision livestock farming evolutionAnimal, 2016
- A survey of multi-agent formation controlAutomatica, 2015
- Behavior‐Based Formation Control of Swarm RobotsMathematical Problems in Engineering, 2014
- Decomposition methods for optimized collision avoidance with multiple threats2011
- A Graph–Based Collision–Free Distributed Formation Control StrategyIFAC Proceedings Volumes, 2011
- Applying Reinforcement Learning to Multi-robot Team Coordination2008
- Operations on Rigid Formations of Autonomous AgentsCommunications in Information and Systems, 2003