NPU-Accelerated Imitation Learning for Thermal Optimization of QoS-Constrained Heterogeneous Multi-Cores
Open Access
- 15 November 2023
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Design Automation of Electronic Systems
- Vol. 29 (1) , 1-23
Abstract
Thermal optimization of a heterogeneous clustered multi-core processor under user-defined QoS targets requires application migration and DVFS. However, selecting the core to execute each application and the VF levels of each cluster is a complex problem because (1) the diverse characteristics and QoS targets of applications require different optimizations, and (2) per-cluster DVFS requires a global optimization considering all running applications. State-of-the-art resource management for power or temperature minimization either relies on measurements that are commonly not available (such as power) or fails to consider all the dimensions of the optimization (e.g., by using simplified analytical models). To solve this, ML methods can be employed. In particular, IL leverages the optimality of an oracle policy, yet at low run-time overhead, by training a model from oracle demonstrations. We are the first to employ IL for temperature minimization under QoS targets. We tackle the complexity by training NN at design time and accelerate the run-time NN inference using NPU. While such NN accelerators are becoming increasingly widespread, they are so far only used to accelerate user applications. In contrast, we use for the first time an existing accelerator on a real platform to accelerate NN-based resource management. To show the superiority of IL compared to RL in our targeted problem, we also develop multi-agent RL-based management. Our evaluation on a HiKey 970 board with an Arm big.LITTLE CPU and NPU shows that IL achieves significant temperature reductions at a negligible run-time overhead. We compare TOP-IL against several techniques. Compared to ondemand Linux governor, TOP-IL reduces the average temperature by up to 17 ˆC at minimal QoS violations for both techniques. Compared to the RL policy, our TOP-IL achieves 63 % to 89 % fewer QoS violations while resulting similar average temperatures. Moreover, TOP-IL outperforms the RL policy in terms of stability. We additionally show that our IL-based technique also generalizes to different software (unseen applications) and even hardware (different cooling) than used for training.Keywords
Funding Information
- Deutsche Forschungsgemeinschaft (146371743—TRR 89)
This publication has 30 references indexed in Scilit:
- Dynamic Resource Management of Heterogeneous Mobile Platforms via Imitation LearningIEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2019
- Dynamic Voltage and Frequency Scaling in NoCs with Supervised and Reinforcement Learning TechniquesIEEE Transactions on Computers, 2018
- Algorithmic Optimization of Thermal and Power Management for Heterogeneous Mobile PlatformsIEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2017
- Cooperative Multi-Agent Reinforcement Learning-Based Co-optimization of Cores, Caches, and On-chip NetworkACM Transactions on Architecture and Code Optimization, 2017
- Profit: Priority and Power/Performance Optimization for Many-Core SystemsIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2017
- Imitation Learning for Dynamic VFI Control in Large-Scale Manycore SystemsIEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2017
- Power Density-Aware Resource Management for Heterogeneous Tiled MulticoresIEEE Transactions on Computers, 2016
- The PARSEC benchmark suite2008
- Reliability concerns in embedded system designsComputer, 2006
- Back-propagation neural networks for modeling complex systemsArtificial Intelligence in Engineering, 1995