Asymptotically Optimal Decentralized Control for Large Population Stochastic Multiagent Systems
Authors:
- 9 September 2008
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automatic Control
- Vol. 53 (7) , 1643-1660
Abstract
The interaction of interest-coupled decision-makers and the uncertainty of individual behavior are prominent characteristics of multiagent systems (MAS). How to break through the framework of conventional control theory, which aims at single decision-maker and single decision objective, and to extend the methodology and tools in the stochastic adaptive control theory to analyze MAS are of great significance. In this paper, a preliminary exploration is made in this direction, and the decentralized control problem is considered for large population stochastic MAS with coupled cost functions. Different from the deterministic discounted costs in the existing differential game models, a time-averaged stochastic cost function is adopted for each agent. The decentralized control law is constructed based on the state aggregation method and tracking-like quadratic optimal control. By using probability limit theory, the stability and optimality of the closed-loop system are analyzed. The main contributions of this paper include the following points. 1) The concepts of asymptotic Nash-equilibrium in probability and almost surely, respectively, are introduced and the relationship between these concepts is illuminated, which provide necessary tools for analyzing the optimality of the decentralized control laws. 2) The closed-loop system is shown to be almost surely uniformly stable, and bounded independently of the number of agents N . 3) The population state average (PSA) is shown to converge to the infinite population mean (IPM) trajectory in the sense of both L2-norm and time average almost surely, as N increases to infinity. 4) The decentralized control law is designed and shown to be almost surely asymptotically optimal; the cost of each agent based on local measurements converges to that based on global measurements almost surely, as N increases to infinity.Keywords
This publication has 35 references indexed in Scilit:
- Large-Population Cost-Coupled LQG Problems With Nonuniform Agents: Individual-Mass Behavior and Decentralized $\varepsilon$-Nash EquilibriaIEEE Transactions on Automatic Control, 2007
- Flocking for Multi-Agent Dynamic Systems: Algorithms and TheoryIEEE Transactions on Automatic Control, 2006
- Uplink Power Adjustment in Wireless Communication Systems: A Stochastic Control AnalysisIEEE Transactions on Automatic Control, 2004
- Information Flow and Cooperative Control of Vehicle FormationsIEEE Transactions on Automatic Control, 2004
- Consensus Problems in Networks of Agents With Switching Topology and Time-DelaysIEEE Transactions on Automatic Control, 2004
- Stable Social Foraging Swarms in a Noisy EnvironmentIEEE Transactions on Automatic Control, 2004
- Stability analysis of swarmsIEEE Transactions on Automatic Control, 2003
- Learning through reinforcement for N-person repeated constrained gamesIEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2002
- Multiuser rate-based flow controlIEEE Transactions on Communications, 1998
- On the existence of equilibria in noncooperative optimal flow controlJournal of the ACM, 1995