Output‐feedback Q‐learning for discrete‐time linear H∞ tracking control: A Stackelberg game approach
- 18 May 2022
- journal article
- research article
- Published by Wiley in International Journal of Robust and Nonlinear Control
- Vol. 32 (12) , 6805-6828
Abstract
In this article, an output‐feedback Q‐learning algorithm is proposed for the discrete‐time linear system to deal with the tracking control problem. The problem is formulated as a zero‐sum game in the Stackelberg game framework with a discount factor to make the value function bounded. According to the principle of optimality, the game algebraic Riccati equation (GARE) is derived and solved by the Q‐learning algorithm to get the optimal solution of the Stackelberg game without requiring the knowledge of system dynamics and state. It is proved that the solution of the algorithm converges to the optimal control input and the worst‐case disturbance with excitation noises during training, and the Stackelberg strategy can achieve a lower disturbance attenuation level than the Nash one. Moreover, the impacts of the discount factor on the stability of the closed‐loop system and solvability of the GARE are analyzed to provide some criteria for the choice of the discount factor. Simulation examples are provided to validate the effectiveness of the algorithm.Keywords
Funding Information
- National Natural Science Foundation of China (T2121002, 62003013, 62173006)
This publication has 42 references indexed in Scilit:
- $ {H}_{ {\infty }}$ Tracking Control of Completely Unknown Continuous-Time Systems via Off-Policy Reinforcement LearningIEEE Transactions on Neural Networks and Learning Systems, 2015
- Optimal Tracking Control of Unknown Discrete-Time Linear Systems Using Input-Output Measured DataIEEE Transactions on Cybernetics, 2015
- Linear Quadratic Tracking Control of Partially-Unknown Continuous-Time Systems Using Reinforcement LearningIEEE Transactions on Automatic Control, 2014
- Reinforcement -learning for optimal tracking control of linear discrete-time systems with unknown dynamicsAutomatica, 2014
- ℋ︁∞ -filtering for singularly perturbed nonlinear systemsInternational Journal of Robust and Nonlinear Control, 2011
- Model-free
H
∞
control design for unknown linear discrete-time systems via Q-learning with LMIAutomatica, 2010
- Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problemAutomatica, 2010
- Adaptive optimal control for continuous-time linear systems based on policy iterationAutomatica, 2009
- Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infinity controlAutomatica, 2007
- The discrete-time Riccati equation related to the H/sub ∞/ control problemIEEE Transactions on Automatic Control, 1994