Abstract
This paper introduces a novel framework that bridges advanced reinforcement learning (RL) with traditional PID control by converting model-based RL policies into interpretable PID gains. By combining inverse reinforcement learning (IRL) with Kullback–Leibler divergence minimization, our method aligns sophisticated control strategies with the simplicity and robustness of PID controllers. In doing so, the proposed approach maintains the transparency and simplicity of PID controllers while incorporating the adaptability, data-driven optimization, and long-horizon planning capabilities of RL. Compatible with both model-based and model-free RL algorithms, the approach has been validated through extensive simulations on benchmark systems and real-world experiments on the Robotarium platform, demonstrating resilience against disturbances, parameter uncertainties, and noise. By blending the strengths of reinforcement learning with the practical familiarity of PID control, the proposed framework offers a data-efficient, scalable, and transparent solution for enhancing PID controller design in complex and dynamic environments.
Funding Information
  • National Key Research and Development Program of China (2024YFE0214000)
  • Natural Science Foundation of Zhejiang Province of China (LRG25F030002)
  • Zhejiang Province leading Geese Plan (2025C01056)

This publication has 51 references indexed in Scilit: