Federated Learning Adaptive Dynamic Programming for Massive Multiagent Mean-Field Games-Based Optimal Consensus
- 22 September 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. 56 (2) , 915-926
Abstract
Massive multiagent systems typically involve a very large number of interactions and conflicts of interest among agents, which presents a significant challenge for achieving stable and efficient adaptive optimal consensus control in real-time. To fill this gap, this article develops a novel federated learning adaptive dynamic programming (FL-ADP) control scheme to solve the massive multiagent mean-field games (MFGs)-based optimal consensus problem. First, the complex interactions of each individual agent with all other agents can be approximated by an average or collective influence in the context of MFGs. Then, a novel undiscounted performance index function involving the mean-field coupling term, the tracking errors and their derivatives is proposed to circumvent the potential impact of the improper discount factor selection and achieve better control performance. By designing the critic–mass neural network structure, the coupled Hamilton–Jacobi–Bellman and Fokker–Planck–Kolmogorov equations are solved to derive the approximate optimal control policy and quantify the probability density function of the collective behavior simultaneously. Additionally, to comply with the required convergence condition of the MFG, a novel event-triggered federated learning mechanism is formulated, which achieves a balance between communication resource consumption and the guarantee of algorithm convergence. On the basis of Lyapunov’s direct method, the tracking errors and the weight estimation errors of all agents are guaranteed to be uniformly ultimately bounded. Simulation results of massive multi-uncrewed aerial vehicle systems affirm the rationality and effectiveness of the proposed method.Keywords
Funding Information
- National Natural Science Foundation of China (62073085)
- Shenzhen Science and Technology Program (JCYJ20230807093513027)
- Postdoctoral Fellowship Program of CPSF (GZC20251178)
This publication has 46 references indexed in Scilit:
- Optimal dynamic output feedback control of unknown linear continuous-time systems by adaptive dynamic programmingAutomatica, 2024
- Fully distributed adaptive control for output consensus of uncertain discrete-time linear multi-agent systemsAutomatica, 2024
- An efficient lightweight convolutional neural network for industrial surface defect detectionArtificial Intelligence Review, 2023
- PMP-based numerical solution for mean field game problem of general nonlinear systemAutomatica, 2022
- Adaptive Dynamic Programming with Applications in Optimal Control2017
- Multi-agent zero-sum differential graphical games for disturbance rejection in distributed controlAutomatica, 2016
- Multi-agent differential graphical games: Online adaptive learning solution for synchronization with optimalityAutomatica, 2012
- Quantized consensusAutomatica, 2007
- Brownian Motion and Stochastic Calculus2007
- Mean field gamesJapanese Journal of Mathematics, 2007