Play It Cool: Dynamic Shifting Prevents Thermal Throttling
Authors:
- Yang ZhouYZYang Zhou
- Feng LiangFLFeng Liang
- Ting-Wu ChinTCTing-Wu Chin
- Diana MarculescuDMDiana Marculescu
Preprint
Abstract
Machine learning (ML) has entered the mobile era where an enormous number of ML models are deployed on edge devices. However, running common ML models on edge devices continuously may generate excessive heat from the computation, forcing the device to "slow down" to prevent overheating, a phenomenon called thermal throttling. This paper studies the impact of thermal throttling on mobile phones: when it occurs, the CPU clock frequency is reduced, and the model inference latency may increase dramatically. This unpleasant inconsistent behavior has a substantial negative effect on user experience, but it has been overlooked for a long time. To counter thermal throttling, we propose to utilize dynamic networks with shared weights and dynamically shift between large and small ML models seamlessly according to their thermal profile, i.e., shifting to a small model when the system is about to throttle. With the proposed dynamic shifting, the application runs consistently without experiencing CPU clock frequency degradation and latency increase. In addition, we also study the resulting accuracy when dynamic shifting is deployed and show that our approach provides a reasonable trade-off between model latency and model accuracy.Keywords
All Related Versions
This publication has 4 references indexed in Scilit:
- Impact of Thermal Throttling on Long-Term Visual Inference in a CPU-Based Edge DeviceElectronics, 2020
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2019
- GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding2018
- Techniques for Multicore Thermal Management: Classification and New Exploration2006