Distributed optimal control of nonlinear multi‐agent systems based on integral reinforcement learning
Authors:
- 4 July 2024
- journal article
- research article
- Published by Wiley in Optimal Control Applications and Methods
- Vol. 45 (6) , 2596-2612
Abstract
In this article, a distributed optimal control approach is proposed for a class of affine nonlinear multi‐agent systems (MASs) with unknown nonlinear dynamics. The game theory is used to formulate the distributed optimal control problem into a differential graphical game problem with synchronized updates of all nodes. A data‐based integral reinforcement learning (IRL) algorithm is used to learn the solution of the coupled Hamilton–Jacobi (HJ) equation without prior knowledge of the drift dynamics, and the actor‐critic neural networks (A‐C NNs) are used to approximate the control law and the cost function, respectively. To update the parameters synchronously, the gradient descent algorithm is used to design the weight update laws of the A‐C NNs. Combining the IRL and the A‐C NNs, a distributed consensus optimal control method is designed. By using the Lyapunov stability theory, the developed optimal control method can show that all signals in the considered system are uniformly ultimately bounded (UUB), and the systems can achieve Nash equilibrium when all agents update their controllers simultaneously. Finally, simulation results are given to illustrate the effectiveness of the developed optimal control approach.Keywords
Funding Information
- National Natural Science Foundation of China (U22A2043)
This publication has 31 references indexed in Scilit:
- Adaptive backstepping control for nonlinear interconnected systems with prespecified-performance-driven output triggeringAutomatica, 2023
- Global optimal consensus for discrete-time multi-agent systems with bounded controlsAutomatica, 2018
- Online reinforcement learning for a class of partially unknown continuous‐time nonlinear systems via value iterationOptimal Control Applications and Methods, 2017
- Multi-agent zero-sum differential graphical games for disturbance rejection in distributed controlAutomatica, 2016
- Multi-agent discrete-time graphical games and reinforcement learning solutionsAutomatica, 2014
- Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systemsAutomatica, 2014
- Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamicsAutomatica, 2012
- Multi-agent differential graphical games: Online adaptive learning solution for synchronization with optimalityAutomatica, 2012
- Optimal Control2012
- Adaptive optimal control for continuous-time linear systems based on policy iterationAutomatica, 2009