Model-free Control Design Using Policy Gradient Reinforcement Learning in LPV Framework
Authors:
- Yajie BaoYBYajie Bao
- Javad Mohammadpour VelniJVJavad Mohammadpour Velni
- 29 June 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 150-155
Abstract
This paper presents an off-policy policy gradient reinforcement learning (RL) approach to control nonlinear systems in linear parameter-varying (LPV) framework. The (parameter-varying) controller is learned from the closed-loop trajectories using off-policy actor-critic methods for RL, instead of being designed based on the system model. To decrease constraint violation (and hence improve safety), exploration around a valid control sequence is proposed to facilitate learning before applying the real policy. Additionally, the valid control sequence is used to determine the complexity of the actor and critic neural networks (NN). The uncertainties in the evolution of LPV scheduling variables are tackled by generating episodes with varying scheduling trajectories. Adapting to unseen scheduling signals is empirically investigated and fine-tuning is used to refine the learned controller. Furthermore, the prior knowledge of the control law (e.g., static state feedback) is embedded in designing the structure of actor NN. Experiments using two numerical examples and a control moment gyroscope (CMG) simulation model illustrate the success of the proposed approach for stabilization and tracking control in the presence of uncertainty in LPV scheduling variables.Keywords
Funding Information
- National Science Foundation
This publication has 14 references indexed in Scilit:
- An Information-Theoretic Approach to Transferability in Task Transfer Learning2019
- Continual lifelong learning with neural networks: A reviewNeural Networks, 2019
- Neural network vehicle models for high-performance automated drivingScience Robotics, 2019
- State-space LPV model identification using kernelized machine learningAutomatica, 2018
- Tube-based LPV constant output reference tracking MPC with error boundIFAC-PapersOnLine, 2017
- LPV state-feedback control of a control moment gyroscopeControl Engineering Practice, 2014
- Stochastic model predictive control of LPV systems via scenario optimizationAutomatica, 2013
- Model structure learning: A support vector machine approach for LPV linear-regression models2011
- Worst-case formulations of model predictive control for systems with bounded parametersAutomatica, 1997
- On the Theory of the Brownian MotionPhysical Review B, 1930