Adaptive dynamic programming and distributionally robust optimal control of linear stochastic system using the Wasserstein metric
- 4 May 2024
- journal article
- research article
- Published by Wiley in International Journal of Adaptive Control and Signal Processing
- Vol. 38 (8) , 2810-2832
Abstract
Summary: In this paper, we consider the optimal control of unknown stochastic dynamical system for both the finite‐horizon and infinite‐horizon cases. The objective of this paper is to find an optimal controller to minimize the expected value of a function which depends on the random disturbance. Throughout this paper, it is assumed that the mean vector and covariance matrix of the disturbance distribution is unknown. An uncertainty set in the space of mean vector and the covariance matrix is introduced. For the finite‐horizon case, we derive a closed‐form expression of the unique optimal policy and the opponents policy that generates the worst‐case distribution. For the infinite‐horizon case, we simplify the Riccati equation obtained in the finite‐hozion setting to an algebraic Riccati equation, which can guarantee the existence of the solution of the Riccati equation. It is shown that the resulting optimal policies obtained in these two cases can stabilize the expected value of the system state under the worst‐case distribution. Furthermore, the unknown system matrices can also be explicitly computed using the adaptive dynamic programming technique, which can help compute the optimal control policy by solving the algebraic Riccati equation. Finally, a simulation example is presented to demonstrate the effectiveness of our theoretical results.Keywords
Funding Information
- Natural Science Foundation of Sichuan Province (2022NSFSC0925)
- National Natural Science Foundation of China (62103341)
This publication has 17 references indexed in Scilit:
- Minimax Q-learning control for linear systems using the Wasserstein metricAutomatica, 2023
- Wasserstein Distributionally Robust Control of Partially Observable Linear Systems: Tractable Approximation and Performance Guarantee2022
- Resilient practical cooperative output regulation for MASs with unknown switching exosystem dynamics under DoS attacksAutomatica, 2022
- Minimax Control of Ambiguous Linear Stochastic Systems Using the Wasserstein Metric2020
- Distributionally robust stochastic control with conic confidence sets2017
- A Convex Optimization Approach to Distributionally Robust Markov Decision Processes With Wasserstein DistanceIEEE Control Systems Letters, 2017
- Robust Linear Quadratic Regulator for uncertain systems2016
- Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamicsAutomatica, 2012
- H∞-Optimal Control and Related Minimax Design Problems2008
- On the numerical solution of the discrete-time algebraic Riccati equationIEEE Transactions on Automatic Control, 1980