Toward Reliable Designs of Data-Driven Reinforcement Learning Tracking Control for Euler-Lagrange Systems
Preprint
Abstract
This paper addresses reinforcement learning based, direct signal tracking control with an objective of developing mathematically suitable and practically useful design approaches. Specifically, we aim to provide reliable and easy to implement designs in order to reach reproducible neural network-based solutions. Our proposed new design takes advantage of two control design frameworks: a reinforcement learning based, data-driven approach to provide the needed adaptation and (sub)optimality, and a backstepping based approach to provide closed-loop system stability framework. We develop this work based on an established direct heuristic dynamic programming (dHDP) learning paradigm to perform online learning and adaptation and a backstepping design for a class of important nonlinear dynamics described as Euler-Lagrange systems. We provide a theoretical guarantee for the stability of the overall dynamic system, weight convergence of the approximating nonlinear neural networks, and the Bellman (sub)optimality of the resulted control policy. We use simulations to demonstrate significantly improved design performance of the proposed approach over the original dHDP.Keywords
All Related Versions
This publication has 34 references indexed in Scilit:
- MRAC for unknown discrete-time nonlinear systems based on supervised neural dynamic programmingNeurocomputing, 2020
- Neural network robust tracking control with adaptive critic framework for uncertain nonlinear systemsNeural Networks, 2018
- Multivalued Robust Tracking Control of Lagrange Systems: Continuous and Discrete-Time AlgorithmsIEEE Transactions on Automatic Control, 2017
- Complete stability analysis of a heuristic approximate dynamic programming control designAutomatica, 2015
- Approximate optimal trajectory tracking for continuous-time nonlinear systemsAutomatica, 2015
- Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learningAutomatica, 2014
- A boundedness result for the direct heuristic dynamic programmingNeural Networks, 2012
- Performance Evaluation of Direct Heuristic Dynamic Programming using Control-Theoretic MeasuresJournal of Intelligent & Robotic Systems, 2009
- Handbook of Learning and Approximate Dynamic Programming2004
- Online learning control by association and reinforcementIEEE Transactions on Neural Networks, 2001