Intelligent Critic Learning for Data-Driven Output Tracking Control With Lightweight Parallelization
- 30 September 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Systems, Man, and Cybernetics: Systems
- Vol. 55 (12) , 8959-8970
Abstract
This article investigates critical challenges in optimal output tracking control, including residual tracking errors, potential instability caused by discount factors, and premature convergence due to inefficient termination criteria. We tackle these issues by developing a data-driven parallel Q-learning algorithm. Specifically, a utility function directly linked to system states is proposed to avoid the instability and error amplification issues in traditional discounted approaches. In addition, the algorithm uses dual lightweight controllers that use convergence properties to enhance learning efficiency. Based on dual controllers, a novel termination criterion is introduced to prevent premature convergence during the training process. Numerical simulations demonstrate that the proposed method eliminates tracking errors, accelerates convergence compared with traditional algorithms, and ensures stable convergence across diverse system dynamics.Keywords
Funding Information
- National Natural Science Foundation of China (62473012, 62222301, 62021003)
- National Science and Technology Major Project (2021ZD0112302, 2021ZD0112301)
- Beijing Natural Science Foundation (F251019)
This publication has 35 references indexed in Scilit:
- Adaptive Optimal Discrete-Time Output-Feedback Using an Internal Model Principle and Adaptive Dynamic ProgrammingIEEE/CAA Journal of Automatica Sinica, 2024
- Dichotomy value iteration with parallel learning design towards discrete-time zero-sum gamesNeural Networks, 2023
- Adaptive Multi-Step Evaluation Design With Stability Guarantee for Discrete-Time Optimal Learning ControlIEEE/CAA Journal of Automatica Sinica, 2023
- Advanced value iteration for discrete-time intelligent critic control: A surveyArtificial Intelligence Review, 2023
- Neural critic learning for tracking control design of constrained nonlinear multi-person zero-sum gamesNeurocomputing, 2022
- Discounted Iterative Adaptive Critic Designs With Novel Stability Analysis for Tracking ControlIEEE/CAA Journal of Automatica Sinica, 2022
- The intelligent critic framework for advanced optimal controlArtificial Intelligence Review, 2022
- Neural-network-based discounted optimal control via an integrated value iteration with accuracy guaranteeNeural Networks, 2021
- A New Integral Critic Learning for Optimal Tracking Control with Applications to Boiler‐Turbine SystemsOptimal Control Applications and Methods, 2021
- A novel adaptive dynamic programming based on tracking error for nonlinear discrete-time systemsAutomatica, 2021