Reinforcement-Learning-Based Dual-Control Methodology for Complex Nonlinear Discrete-Time Systems With Application to Spark Engine EGR Operation
Authors:
- Brian C. KaulBKBrian C. Kaul
- S. JagannathanS. Jagannathan
- James A. DrallmeierJDJames A. Drallmeier
- 15 July 2008
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks
- Vol. 19 (8) , 1369-1388
Abstract
A novel reinforcement-learning-based dual-control methodology adaptive neural network (NN) controller is developed to deliver a desired tracking performance for a class of complex feedback nonlinear discrete-time systems, which consists of a second-order nonlinear discrete-time system in nonstrict feedback form and an affine nonlinear discrete-time system, in the presence of bounded and unknown disturbances. For example, the exhaust gas recirculation (EGR) operation of a spark ignition (SI) engine is modeled by using such a complex nonlinear discrete-time system. A dual-controller approach is undertaken where primary adaptive critic NN controller is designed for the nonstrict feedback nonlinear discrete-time system whereas the secondary one for the affine nonlinear discrete-time system but the controllers together offer the desired performance. The primary adaptive critic NN controller includes an NN observer for estimating the states and output, an NN critic, and two action NNs for generating virtual control and actual control inputs for the nonstrict feedback nonlinear discrete-time system, whereas an additional critic NN and an action NN are included for the affine nonlinear discrete-time system by assuming the state availability. All NN weights adapt online towards minimization of a certain performance index, utilizing gradient-descent-based rule. Using Lyapunov theory, the uniformly ultimate boundedness (UUB) of the closed-loop tracking error, weight estimates, and observer estimates are shown. The adaptive critic NN controller performance is evaluated on an SI engine operating with high EGR levels where the controller objective is to reduce cyclic dispersion in heat release while minimizing fuel intake. Simulation and experimental results indicate that engine out emissions drop significantly at 20% EGR due to reduction in dispersion in heat release thus verifying the dual-control approach.Keywords
This publication has 15 references indexed in Scilit:
- Neural Network Controller Development and Implementation for Spark Ignition Engines With High EGR LevelsIEEE Transactions on Neural Networks, 2007
- Neural network-based output feedback controller for lean operation of spark ignition engines2006
- Reinforcement Learning-Based Output Feedback Control of Nonlinear Systems With Input ConstraintsIEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2005
- Adaptive dynamic programmingIEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews), 2002
- Adaptive output feedback control of uncertain nonlinear systems using single-hidden-layer neural networksIEEE Transactions on Neural Networks, 2002
- Online learning control by association and reinforcementIEEE Transactions on Neural Networks, 2001
- A separation principle for the stabilization of a class of nonlinear systemsIEEE Transactions on Automatic Control, 1999
- Observing and modeling nonlinear dynamics in an internal combustion enginePhysical Review E, 1998
- Adaptive control of a class of nonlinear discrete-time systems using neural networksIEEE Transactions on Automatic Control, 1995
- Stochastic choice of basis functions in adaptive function approximation and the functional-link netIEEE Transactions on Neural Networks, 1995