A Novel PID Design Method via Model-Based Reinforcement Learning Algorithms
- 7 April 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 23 (15455955) , 8059-8071
Abstract
This paper introduces a novel framework that bridges advanced reinforcement learning (RL) with traditional PID control by converting model-based RL policies into interpretable PID gains. By combining inverse reinforcement learning (IRL) with Kullback–Leibler divergence minimization, our method aligns sophisticated control strategies with the simplicity and robustness of PID controllers. In doing so, the proposed approach maintains the transparency and simplicity of PID controllers while incorporating the adaptability, data-driven optimization, and long-horizon planning capabilities of RL. Compatible with both model-based and model-free RL algorithms, the approach has been validated through extensive simulations on benchmark systems and real-world experiments on the Robotarium platform, demonstrating resilience against disturbances, parameter uncertainties, and noise. By blending the strengths of reinforcement learning with the practical familiarity of PID control, the proposed framework offers a data-efficient, scalable, and transparent solution for enhancing PID controller design in complex and dynamic environments.Keywords
Funding Information
- National Key Research and Development Program of China (2024YFE0214000)
- Natural Science Foundation of Zhejiang Province of China (LRG25F030002)
- Zhejiang Province leading Geese Plan (2025C01056)
This publication has 51 references indexed in Scilit:
- PID Controllers2026
- Probabilistic design of optimal sequential decision-making algorithms in learning and controlAnnual Reviews in Control, 2022
- Reinforcement learning based adaptive PID controller design for control of linear/nonlinear unstable processesApplied Soft Computing, 2022
- Design of a Reinforcement Learning PID ControllerIEEJ Transactions on Electrical and Electronic Engineering, 2021
- $\mathsf {SafePILCO}$: A Software Tool for Safe and Data-Efficient Policy Synthesis2020
- Inverse optimal control of stochastic systems driven by Lévy processesAutomatica, 2019
- Iterative learning control based on extremum seekingAutomatica, 2016
- PID Control in the Third Millennium2012
- Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problemAutomatica, 2010
- Gaussian Processes in Machine Learning2004