Autonomous PID Tuning: Two-Phase Reinforcement Learning Through Adversarial Imitation Learning Under Imperfect Demonstrations

Authors:
Publisher website
Google Scholar
Abstract
This paper presents stable reinforcement learning through adversarial imitation learning (SRAIL), a two-phase learning framework for automatic proportional integral derivative (PID) tuning in the absence of high-quality demonstration data. First, adversarial imitation learning (AIL) is used to learn an initial control policy using imperfect demonstration data. Subsequently, reinforcement learning (RL) refines this initial policy to derive the optimal PID control policy. SRAIL is designed to ensure stable and safe exploration of the RL agent, even with imperfect demonstration data, thereby enabling efficient and automatic tuning in industrial environments. Conventional PID tuning methods are time-consuming owing to system complexity and lack of prior knowledge. Furthermore, the random exploratory nature of RL presents significant risks during the early training phase, limiting its practical application. SRAIL addresses these challenges by ensuring stable control performance despite the absence of high-quality demonstration data and by enhancing the convergence speed of RL. The experimental results demonstrate that SRAIL minimizes random exploration in the initial learning phase, ensuring safety, and highlighting its potential for implementation of RL in real-world industrial environments.
Funding Information
  • National Research Foundation of Korea
  • Korean Government through Ministry of Science and ICT (NRF-2022R1A2C2004457)
  • Brain Korea 21 FOUR
  • Ministry of Education of Korea and the National Research Foundation of Korea

This publication has 32 references indexed in Scilit: