Integrated Fault Detection and Fault-Tolerant Optimal Tracking Control for Unknown Nonlinear Systems With State and Input Constraints Using Safe Reinforcement Learning
- 30 October 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Systems, Man, and Cybernetics: Systems
- Vol. 56 (1) , 46-58
Abstract
This article deals with the problem of fault detection (FD) and fault-tolerant optimal control in unknown affine nonlinear systems subject to actuator and process faults with state and input constraints. In this regard, a novel integrated fault-tolerant optimal tracking control (FTOTC) scheme is proposed based on safe reinforcement learning (RL) without requiring prior knowledge of the system dynamics. First, by introducing a discounted cost function that incorporates a control barrier function (CBF) term, the constrained FTOTC problem is transformed into an unconstrained optimal regulation problem for an augmented system formed from the dynamics of the tracking error and the reference trajectory. Subsequently, we adopt an identifier-critic framework to solve the associated Hamilton–Jacobi-Bellman (HJB) equation online, training both neural networks (NNs) simultaneously. We propose an experience replay (ER)-enhanced update law for the identifier NN, where the forgetting factor (FF) is considered to be variable and dependent on the state estimation errors and measurement noise estimation. It is shown that the proposed identifier update law improves the convergence rate and reduces NN weight estimation error in comparison with the constant FF condition without the ER enhancement, while increasing robustness against measurement noise. Distinct from typical RL algorithms, the initial admissible control condition is eliminated by using a modified variable gain gradient descent-based update law for the critic NN, which leverages a stabilizing term. The learning rate of critic update law is a function of instantaneous HJB error, yielding a tighter a tighter residual error bound for the NN weights. We prove that all the system states and the identifier and critic NN weight errors are uniformly ultimately bounded (UUB) by the Lyapunov stability theory. Simulation results on a single-link robotic manipulator demonstrate the effectiveness of the presented integrated scheme in different fault scenarios.Keywords
This publication has 41 references indexed in Scilit:
- Safe reinforcement learning: A control barrier function optimization approachInternational Journal of Robust and Nonlinear Control, 2020
- Robust adaptive finite‐time parameter estimation and control for robotic systemsInternational Journal of Robust and Nonlinear Control, 2014
- Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learningAutomatica, 2014
- Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systemsAutomatica, 2014
- Fault-tolerant control systems: A comparative study between active and passive approachesAnnual Reviews in Control, 2012
- Optimal Control2012
- Supervisory fault tolerant control for a class of uncertain nonlinear systemsAutomatica, 2009
- Bibliographical review on reconfigurable fault-tolerant control systemsAnnual Reviews in Control, 2008
- Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approachAutomatica, 2005
- Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations1997