Deep Reinforcement Learning Based Dynamic Proportional-Integral (PI) Gain Auto-Tuning Method for a Robot Driver System
Open Access
- 16 March 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 10, 31043-31057
Abstract
To meet the growing trend of stringent fuel economy regulations, automakers around the world are designing modules such as engines, motors, transmissions and batteries to be as efficient as possible. In order to verify the effect of these designs on the overall fuel efficiency of the vehicle, the vehicle equipped with each module is placed on the chassis dynamometer, driven to follow the target vehicle speed, and actual fuel efficiency is measured. These tests are traditionally performed by human operators, but are now being replaced by robots (physical or software) to ensure the accuracy and reliability of test results. Although the conventionally proposed proportional integral (PI)-based controller has a simple structure and is easy to implement, it requires the process of finding the optimal gain whenever the test conditions such as vehicle or drive cycle change, which is difficult and time consuming. In this study, we propose a proportional integral controller gain adjustment algorithm using deep reinforcement learning. The reinforcement learning agent learns to dynamically modify the PI gain value of the acceleration/deceleration pedal to better follow the target vehicle in a simulation environment. The perturbation is used in each training episode to reduce the difference between the simulation and real testing environment. Upon completion of the training process, the trained agent performs an adjustment process that generates a reference gain table. We then use this reference gain table to perform a real test. The performance of the proposed system was evaluated using Hyundai Tucson HEV (NX4) on an AVL chassis dynamometer. We also compared the performance of our proposed algorithm to traditional fuzzy logic-based PI controllers. The obtained experimental results show that the proposed control system achieved a performance improvement of aounrd 46.8% compared to the conventional PI control system in terms of root mean square error.Keywords
Funding Information
- Hyundai Motor Group, Korea Institute of Police Technology
- Korean Government [Korea National Police Agency (KNPA)] (092021C26S03000)
- Artificial Intelligence (AI)-based Autonomous Driving Computing Module Development and Service Demonstration Program
- Ministry of Trade, Industry, and Energy of Korea (20005673)
- Brain Korea 21 (BK21) Program through the National Research Foundation of Korea
- Ministry of Education (5199990814084)
This publication has 28 references indexed in Scilit:
- Speed Tracking Control of a Vehicle Robot Driver System Using Multiple Sliding Surface Control SchemesInternational Journal of Advanced Robotic Systems, 2013
- Identification and adaptive neural network control of a DC motor system with dead-zone characteristicsISA Transactions, 2011
- A Comparison of PID Controller Tuning MethodsThe Canadian Journal of Chemical Engineering, 2005
- Adaptive recurrent neural network control of biological wastewater treatmentInternational Journal of Intelligent Systems, 2004
- Applying neural networks to on-line updated PID controllers for nonlinear process controlJournal of Process Control, 2004
- Simple analytic rules for model reduction and PID controller tuningJournal of Process Control, 2003
- PI/PID Controller Design Based on Direct Synthesis and Disturbance RejectionIndustrial & Engineering Chemistry Research, 2002
- Automotive test drive cycles for emission measurement and real-world emission levels-a reviewProceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering, 2002
- PID controller tuning for desired closed‐loop responses for SI/SO systemsAIChE Journal, 1998
- Internal model control: PID controller designIndustrial & Engineering Chemistry Process Design and Development, 1986