Coordinated DVFS and Precision Control for Deep Neural Networks
- 17 September 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Architecture Letters
- Vol. 18 (2) , 136-140
Abstract
Traditionally, DVFS has been the main mechanism to trade-off performance and power. We observe that Deep Neural Network (DNN) applications offer the possibility to trade-off performance, power, and accuracy using both DVFS and numerical precision levels. Our proposed approach, Power-Inference accuracy Trading (PIT), monitors server's load, and accordingly adjusts the precision of the DNN model and the DVFS setting of GPU to trade-off accuracy and power consumption with response time. At high loads and tight request arrivals, PIT leverages INT8-precision instructions of GPU to dynamically change the precision of deployed DNN models and boosts GPU frequency to execute requests faster at the expense of accuracy reduction and high power consumption. However, when requests' arrival rate is relaxed and there is slack time for requests, PIT deploys high precision version of models to improve accuracy and reduces GPU frequency to decrease power consumption. We implement and deploy PIT on a state-of-the-art server equipped with a Tesla P40 GPU. Experimental results demonstrate that depending on load, PIT can improve response time up to 11% compared to a job scheduler that uses only FP32 precision. It also improves the energy consumption by up to 28%, while achieving around 99.5% accuracy of sole FP32-precision.Keywords
Funding Information
- NSF (1814920)
- DoD ARO (W911NF-19-1-0484)
This publication has 10 references indexed in Scilit:
- Incremental-Precision Based Feature Computation and Multi-Level Classification for Low-Energy Internet-of-ThingsIEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2018
- 3Sigma2018
- CANNA: Neural network acceleration using configurable approximation on GPGPU2018
- Swayam2017
- Deep Learning with Low Precision by Half-Wave Gaussian Quantization2017
- In-Datacenter Performance Analysis of a Tensor Processing Unit2017
- Going deeper with convolutions2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Deep convolutional neural networks for LVCSR2013
- SLURM: Simple Linux Utility for Resource Management2003