Feiran Zhao
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Feiran Zhao has a total of 24 co-authors. Here are the top co-authors by number of shared publications:
- Keyou You
Tsinghua University
15 shared publications - Florian Dörfler
ETH Zurich, Zurich, Switzerland
4 shared publications - Alessandro Chiuso
Department of Information Engineering, University of Padova, Padova, Italy
2 shared publications Laisheng Tong
CRRC Zhuzhou Locomotive Co. Ltd., Zhuzhou, China
2 shared publications- Shiji Song
Tsinghua University
2 shared publications - Tamer Başar
Department of Electrical and Computer Engineering, University of Illinois Urbana-Champaign, Urbana, IL, USA
2 shared publications
Latest Publications
- Top Cited Papers
- 1 July 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Control Systems Technology
- Vol. 34 (4) , 2122-2129
An Adaptive Data-Enabled Policy Optimization Approach for Autonomous Bicycle Control
- 1 July 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Control Systems Technology
- Vol. 34 (4) , 2122-2129
Top Cited PapersAn Adaptive Data-Enabled Policy Optimization Approach for Autonomous Bicycle Control
- 1 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automatic Control
- Vol. 70 (12) , 8414-8421
Policy Gradient Methods for the Cost-Constrained LQR: Strong Duality and Global Convergence
- 1 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automatic Control
- Vol. 70 (12) , 8414-8421
Policy Gradient Methods for the Cost-Constrained LQR: Strong Duality and Global Convergence
- 11 June 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Control Systems Letters
- Vol. 9, 961-966
Regularization for Covariance Parameterization of Direct Data-Driven LQR Control
As the benchmark of data-driven control methods, the linear quadratic regulator (LQR) problem has gained significant attention. A growing trend is direct LQR design, which finds the optimal LQR gain directly from raw data and bypassing system identification. To achieve this, our previous work develops a direct LQR formulation parameterized by sample covariance. In this paper, we propose a regularization method for the covariance-parameterized LQR. We show that the regularizer accounts for the uncertainty in both the steady-state covariance matrix corresponding to closed-loop stability, and the LQR cost function corresponding to averaged control performance. With a positive or negative coefficient, the regularizer can be interpreted as promoting either exploitation or exploration, which are well-known trade-offs in reinforcement learning. In simulations, we observe that our covariance-parameterized LQR with regularization can significantly outperform the certainty-equivalence LQR in terms of both the optimality gap and the robust stability.
- 11 June 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Control Systems Letters
- Vol. 9, 961-966
Regularization for Covariance Parameterization of Direct Data-Driven LQR Control
As the benchmark of data-driven control methods, the linear quadratic regulator (LQR) problem has gained significant attention. A growing trend is direct LQR design, which finds the optimal LQR gain directly from raw data and bypassing system identification. To achieve this, our previous work develops a direct LQR formulation parameterized by sample covariance. In this paper, we propose a regularization method for the covariance-parameterized LQR. We show that the regularizer accounts for the uncertainty in both the steady-state covariance matrix corresponding to closed-loop stability, and the LQR cost function corresponding to averaged control performance. With a positive or negative coefficient, the regularizer can be interpreted as promoting either exploitation or exploration, which are well-known trade-offs in reinforcement learning. In simulations, we observe that our covariance-parameterized LQR with regularization can significantly outperform the certainty-equivalence LQR in terms of both the optimality gap and the robust stability.
- Top Cited Papers
- 13 May 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automatic Control
- Vol. 70 (11) , 7217-7232
Data-Enabled Policy Optimization for Direct Adaptive Learning of the LQR
Direct data-driven design methods for the linear quadratic regulator (LQR) mainly use offline or episodic data batches, and their online adaptation remains unclear. In this article, we propose a direct adaptive method to learn the LQR from online closed-loop data. First, we propose a new policy parameterization based on the sample covariance to formulate a direct data-driven LQR problem, which is shown to be equivalent to the certainty-equivalence LQR with optimal nonasymptotic guarantees. Second, we design a novel data-enabled policy optimization (DeePO) method to directly update the policy, where the gradient is explicitly computed using only a batch of persistently exciting (PE) data. Third, we establish its global convergence via a projected gradient dominance property. Importantly, we efficiently use DeePO to adaptively learn the LQR by performing only one-step projected gradient descent per sample of the closed-loop system, which also leads to an explicit recursive update of the policy. Under PE inputs and for bounded noise, we show that the average regret of the LQR cost is upper bounded by two terms signifying a sublinear decrease in time $\mathcal {O}(1/\sqrt{T})$ plus a bias scaling inversely with signal-to-noise ratio, which are independent of the noise statistics. Finally, we perform simulations to validate the theoretical results and demonstrate the computational and sample efficiency of our method.
- 13 May 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automatic Control
- Vol. 70 (11) , 7217-7232
Top Cited PapersData-Enabled Policy Optimization for Direct Adaptive Learning of the LQR
Direct data-driven design methods for the linear quadratic regulator (LQR) mainly use offline or episodic data batches, and their online adaptation remains unclear. In this article, we propose a direct adaptive method to learn the LQR from online closed-loop data. First, we propose a new policy parameterization based on the sample covariance to formulate a direct data-driven LQR problem, which is shown to be equivalent to the certainty-equivalence LQR with optimal nonasymptotic guarantees. Second, we design a novel data-enabled policy optimization (DeePO) method to directly update the policy, where the gradient is explicitly computed using only a batch of persistently exciting (PE) data. Third, we establish its global convergence via a projected gradient dominance property. Importantly, we efficiently use DeePO to adaptively learn the LQR by performing only one-step projected gradient descent per sample of the closed-loop system, which also leads to an explicit recursive update of the policy. Under PE inputs and for bounded noise, we show that the average regret of the LQR cost is upper bounded by two terms signifying a sublinear decrease in time $\mathcal {O}(1/\sqrt{T})$ plus a bias scaling inversely with signal-to-noise ratio, which are independent of the noise statistics. Finally, we perform simulations to validate the theoretical results and demonstrate the computational and sample efficiency of our method.
- 1 May 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Aerospace and Electronic Systems
- Vol. PP (00189251) , 1-12
Unified Aeroelastic Flutter and Loads Control via Data-Enabled Policy Optimization
Ultra-efficient, high-aspect-ratio wings offer a promising solution for reducing emissions in next-generation aircraft. However, these designs are sensitive to atmospheric disturbances and prone to instability. While active control strategies can mitigate structural loads and stabilize the system, their development is challenging due to the uncertain and timevarying nature of aeroelastic systems. This paper addresses these challenges with a direct, adaptive, data-driven approach. The proposed data-enabled policy optimization algorithm leverages sample covariance to directly learn and adapt control strategies from a single batch of persistently exciting, closed-loop inputoutput data. A forgetting factor mechanism enhances adaptability to time-varying dynamics during operation. The algorithm is explicit and recursive, requiring only a single step of projected gradient descent per sample, improving computational efficiency and enabling real-time application. Numerical simulations demonstrate that the proposed algorithm effectively suppresses unstable flutter, alleviates structural loads, adapts to dynamic time variations, and minimizes control effort—all without requiring prior knowledge of system dynamics or disturbances.
- 1 May 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Aerospace and Electronic Systems
- Vol. PP (00189251) , 1-12
Unified Aeroelastic Flutter and Loads Control via Data-Enabled Policy Optimization
Ultra-efficient, high-aspect-ratio wings offer a promising solution for reducing emissions in next-generation aircraft. However, these designs are sensitive to atmospheric disturbances and prone to instability. While active control strategies can mitigate structural loads and stabilize the system, their development is challenging due to the uncertain and timevarying nature of aeroelastic systems. This paper addresses these challenges with a direct, adaptive, data-driven approach. The proposed data-enabled policy optimization algorithm leverages sample covariance to directly learn and adapt control strategies from a single batch of persistently exciting, closed-loop inputoutput data. A forgetting factor mechanism enhances adaptability to time-varying dynamics during operation. The algorithm is explicit and recursive, requiring only a single step of projected gradient descent per sample, improving computational efficiency and enabling real-time application. Numerical simulations demonstrate that the proposed algorithm effectively suppresses unstable flutter, alleviates structural loads, adapts to dynamic time variations, and minimizes control effort—all without requiring prior knowledge of system dynamics or disturbances.
- 17 February 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automatic Control
- Vol. 70 (7) , 4920-4927
Asynchronous Parallel Policy Gradient Methods for the Linear Quadratic Regulator
Learning policies in an asynchronous parallel way is essential to numerous successes of RL for solving complex problems. However, their convergence has not been rigorously evaluated. To improve the theoretical understanding, we adopt the asynchronous parallel zero-order policy gradient (AZOPG) method to solve the continuous-time linear quadratic regulation problem. Specifically, multiple workers independently perform system rollouts to estimate zero-order PGs, which are then aggregated in a master for policy updates. Moreover, each worker is allowed to interact with the master asynchronously, leading to delayed PG estimates. By quantifying the convergence rate of AZOPG, we show a linear speedup property both in theory and simulation, which reveals the advantages of using asynchronous parallel workers in learning policies.
- 17 February 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automatic Control
- Vol. 70 (7) , 4920-4927
Asynchronous Parallel Policy Gradient Methods for the Linear Quadratic Regulator
Learning policies in an asynchronous parallel way is essential to numerous successes of RL for solving complex problems. However, their convergence has not been rigorously evaluated. To improve the theoretical understanding, we adopt the asynchronous parallel zero-order policy gradient (AZOPG) method to solve the continuous-time linear quadratic regulation problem. Specifically, multiple workers independently perform system rollouts to estimate zero-order PGs, which are then aggregated in a master for policy updates. Moreover, each worker is allowed to interact with the master asynchronously, leading to delayed PG estimates. By quantifying the convergence rate of AZOPG, we show a linear speedup property both in theory and simulation, which reveals the advantages of using asynchronous parallel workers in learning policies.
- 6 September 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automatic Control
- Vol. 70 (3) , 1455-1466
Convergence and Sample Complexity of Policy Gradient Methods for Stabilizing Linear Systems
System stabilization via policy gradient (PG) methods has drawn increasing attention in both control and machine learning communities. In this paper, we study their convergence and sample complexity for stabilizing linear time-invariant systems in terms of the number of system rollouts. Our analysis is built upon a discounted linear quadratic regulator (LQR) method which alternatively updates the policy and the discount factor of the LQR problem. Firstly, we propose an explicit rule to adaptively adjust the discount factor by exploring the stability margin of a linear control policy. Then, we establish the sample complexity of PG methods for stabilization, which only adds a coefficient logarithmic in the spectral radius of the state matrix to that for solving the LQR problem with a prior stabilizing policy. Finally, we perform simulations to validate our theoretical findings and demonstrate the effectiveness of our method on a class of nonlinear systems
- 6 September 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automatic Control
- Vol. 70 (3) , 1455-1466
Convergence and Sample Complexity of Policy Gradient Methods for Stabilizing Linear Systems
System stabilization via policy gradient (PG) methods has drawn increasing attention in both control and machine learning communities. In this paper, we study their convergence and sample complexity for stabilizing linear time-invariant systems in terms of the number of system rollouts. Our analysis is built upon a discounted linear quadratic regulator (LQR) method which alternatively updates the policy and the discount factor of the LQR problem. Firstly, we propose an explicit rule to adaptively adjust the discount factor by exploring the stability margin of a linear control policy. Then, we establish the sample complexity of PG methods for stabilization, which only adds a coefficient logarithmic in the spectral radius of the state matrix to that for solving the LQR problem with a prior stabilizing policy. Finally, we perform simulations to validate our theoretical findings and demonstrate the effectiveness of our method on a class of nonlinear systems
- Open Access
Harnessing Data for Accelerating Model Predictive Control by Constraint Removal
Model predictive control (MPC) solves a receding-horizon optimization problem in real-time, which can be computationally demanding when there are thousands of constraints. To accelerate online computation of MPC, we utilize data to adaptively remove the constraints while maintaining the MPC policy unchanged. Specifically, we design the removal rule based on the Lipschitz continuity of the MPC policy. This removal rule can use the information of historical data according to the Lipschitz constant and the distance between the current state and historical states. In particular, we provide the explicit expression for calculating the Lipschitz constant by the model parameters. Finally, simulations are performed to validate the effectiveness of the proposed method.
Open AccessHarnessing Data for Accelerating Model Predictive Control by Constraint Removal
Model predictive control (MPC) solves a receding-horizon optimization problem in real-time, which can be computationally demanding when there are thousands of constraints. To accelerate online computation of MPC, we utilize data to adaptively remove the constraints while maintaining the MPC policy unchanged. Specifically, we design the removal rule based on the Lipschitz continuity of the MPC policy. This removal rule can use the information of historical data according to the Lipschitz constant and the distance between the current state and historical states. In particular, we provide the explicit expression for calculating the Lipschitz constant by the model parameters. Finally, simulations are performed to validate the effectiveness of the proposed method.
- 27 February 2024
- Published by Springer Nature in Journal of Systems Science and Complexity
- Vol. 37 (1) , 152-168
Data-Driven Control of Linear Systems via Quantized Feedback
- 27 February 2024
- Published by Springer Nature in Journal of Systems Science and Complexity
- Vol. 37 (1) , 152-168
Data-Driven Control of Linear Systems via Quantized Feedback
- Open Access
Data-enabled Policy Optimization for the Linear Quadratic Regulator
Policy optimization (PO), an essential approach of reinforcement learning for a broad range of system classes, requires significantly more system data than indirect (identification-followed-by-control) methods or behavioral-based direct methods even in the simplest linear quadratic regulator (LQR) problem. In this paper, we take an initial step towards bridging this gap by proposing the data-enabled policy optimization (DeePO) method, which requires only a finite number of sufficiently exciting data to iteratively solve the LQR problem via PO. Based on a data-driven closed-loop parameterization, we are able to directly compute the policy gradient from a batch of persistently exciting data. Next, we show that the nonconvex PO problem satisfies a projected gradient dominance property by relating it to an equivalent convex program, leading to the global convergence of DeePO. Moreover, we apply regularization methods to enhance certainty-equivalence and robustness of the resulting controller and show an implicit regularization property. Finally, we perform simulations to validate our results.
Open AccessData-enabled Policy Optimization for the Linear Quadratic Regulator
Policy optimization (PO), an essential approach of reinforcement learning for a broad range of system classes, requires significantly more system data than indirect (identification-followed-by-control) methods or behavioral-based direct methods even in the simplest linear quadratic regulator (LQR) problem. In this paper, we take an initial step towards bridging this gap by proposing the data-enabled policy optimization (DeePO) method, which requires only a finite number of sufficiently exciting data to iteratively solve the LQR problem via PO. Based on a data-driven closed-loop parameterization, we are able to directly compute the policy gradient from a batch of persistently exciting data. Next, we show that the nonconvex PO problem satisfies a projected gradient dominance property by relating it to an equivalent convex program, leading to the global convergence of DeePO. Moreover, we apply regularization methods to enhance certainty-equivalence and robustness of the resulting controller and show an implicit regularization property. Finally, we perform simulations to validate our results.