A Physics-Informed Neural Networks Framework to Solve the Infinite-Horizon Optimal Control Problem
- 13 December 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 3, 6014-6019
Abstract
In this work, we leverage physics-informed neural networks (PINNs) to approximately solve the infinite-horizon optimal control problem for nonlinear systems. Specifically, since PINNs are generally able to solve a class of partial differential equations, they can be employed to approximate the value function in the infinite-horizon optimal control problem, via solving the associated steady-state Hamilton-Jacobi-Bellman (HJB) equation. However, the issue with such a direct approach is that the steady HJB equation generally yields more than one solution, hence directly employing PINNs to solve it can lead to divergence of the method. To tackle this problem, we instead apply PINNs to a finite-horizon variant of the steady-state HJB equation which has a unique solution, and which uniformly approximates the infinite-horizon optimal value function as the horizon increases. A method to verify whether the selected horizon is large enough is also provided, as well as an algorithm to increase it with reduced computations if it is not. Unlike conventional methods, the proposed approach does not require knowledge of a stabilizing controller, the execution of computationally expensive iterations, or polynomial basis functions for approximation.Keywords
All Related Versions
Funding Information
- NSF (CAREER CPS-1851588,S&AS-1849198,SATC-1801611)
This publication has 16 references indexed in Scilit:
- Deep Neural Network-Based Approximate Optimal Tracking for Unknown Nonlinear SystemsIEEE Transactions on Automatic Control, 2023
- Model-Based Policy Iterations for Nonlinear Systems via Controlled Hamiltonian DynamicsIEEE Transactions on Automatic Control, 2022
- Data-based reinforcement learning approximate optimal control for an uncertain nonlinear system with control effectiveness faultsAutomatica, 2020
- Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equationsJournal of Computational Physics, 2019
- Efficient model-based reinforcement learning for approximate online optimal controlAutomatica, 2016
- Adaptive Optimal Control of Unknown Constrained-Input Systems Using Policy Iteration and Neural NetworksIEEE Transactions on Neural Networks and Learning Systems, 2013
- Optimal Control2012
- Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problemAutomatica, 2010
- Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approachAutomatica, 2005
- Approximate Solutions to the Time-Invariant Hamilton–Jacobi–Bellman EquationJournal of Optimization Theory and Applications, 1998