Abstract
This article deals with the problem of fault detection (FD) and fault-tolerant optimal control in unknown affine nonlinear systems subject to actuator and process faults with state and input constraints. In this regard, a novel integrated fault-tolerant optimal tracking control (FTOTC) scheme is proposed based on safe reinforcement learning (RL) without requiring prior knowledge of the system dynamics. First, by introducing a discounted cost function that incorporates a control barrier function (CBF) term, the constrained FTOTC problem is transformed into an unconstrained optimal regulation problem for an augmented system formed from the dynamics of the tracking error and the reference trajectory. Subsequently, we adopt an identifier-critic framework to solve the associated Hamilton–Jacobi-Bellman (HJB) equation online, training both neural networks (NNs) simultaneously. We propose an experience replay (ER)-enhanced update law for the identifier NN, where the forgetting factor (FF) is considered to be variable and dependent on the state estimation errors and measurement noise estimation. It is shown that the proposed identifier update law improves the convergence rate and reduces NN weight estimation error in comparison with the constant FF condition without the ER enhancement, while increasing robustness against measurement noise. Distinct from typical RL algorithms, the initial admissible control condition is eliminated by using a modified variable gain gradient descent-based update law for the critic NN, which leverages a stabilizing term. The learning rate of critic update law is a function of instantaneous HJB error, yielding a tighter a tighter residual error bound for the NN weights. We prove that all the system states and the identifier and critic NN weight errors are uniformly ultimately bounded (UUB) by the Lyapunov stability theory. Simulation results on a single-link robotic manipulator demonstrate the effectiveness of the presented integrated scheme in different fault scenarios.