A Deep Q-Learning Direct Torque Controller for Permanent Magnet Synchronous Motors
Open Access
- 26 April 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Open Journal of the Industrial Electronics Society
- Vol. 2, 388-400
Abstract
Torque control of electric drives is a challenging task, as high dynamics need to be achieved despite different input and state constraints while also pursuing secondary objectives, e.g., maximizing power efficiency. Whereas most state-of-the-art methods generally necessitate thorough knowledge about the system model, a model-free deep reinforcement learning torque controller is proposed. In particular, the deep Q-learning algorithm is utilized which has been successfully used in different application scenarios with a finite action set in the recent past. This nicely fits the considered system, a permanent magnet synchronous motor supplied by a two-level voltage source inverter, since the latter is a power supply unit with a limited amount of distinct switching states. This contribution investigates the deep Q-learning finite control set framework and its design, including the conception of a reward function that incorporates the demands concerning torque tracking, efficiency maximization and compliance with operation limits. In addition, a comprehensive hyperparameter optimization is presented, which addresses the many degrees of freedom of the deep Q-learning algorithm striving for an optimal controller configuration. Advantages and remaining challenges of the proposed algorithm are disclosed through an extensive validation, which includes a direct comparison with a state-of-the-art model predictive direct torque controller.Keywords
Funding Information
- Paderborn Center for Parallel Computing
This publication has 22 references indexed in Scilit:
- Long-Horizon Direct Model Predictive Control: Modified Sphere Decoding for Transient OperationIEEE Transactions on Industry Applications, 2018
- Analytical computation of the optimal reference currents for MTPC/MTPA, MTPV and MTPF operation of anisotropic synchronous machines considering stator resistance and mutual inductance2017
- Human-level control through deep reinforcement learningNature, 2015
- Model Predictive Direct Torque Control With Finite Control Set for PMSM Drive Systems, Part 1: Maximum Torque Per Ampere OperationIEEE Transactions on Industrial Informatics, 2013
- Model Predictive Direct Torque Control With Finite Control Set for PMSM Drive Systems, Part 2: Field Weakening OperationIEEE Transactions on Industrial Informatics, 2012
- Computationally Efficient Model Predictive Direct Torque ControlIEEE Transactions on Power Electronics, 2011
- Advanced Electrical Drives2011
- A New Quick-Response and High-Efficiency Control Strategy of an Induction MotorIEEE Transactions on Industry Applications, 1986
- Field-Oriented Control of a Standard AC Motor Using MicroprocessorsIEEE Transactions on Industry Applications, 1980
- On the Theory of Dynamic ProgrammingProceedings of the National Academy of Sciences, 1952