Neural Network Inference on Mobile SoCs
- 20 January 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Design & Test
- Vol. 37 (5) , 50-57
Abstract
The ever-increasing demand from mobile Machine Learning (ML) applications calls for evermore powerful on-chip computing resources. Mobile devices are empowered with heterogeneous multi-processor Systems-on-Chips (SoCs) to process ML workloads such as Convolutional Neural Network (CNN) inference. Mobile SoCs house several different types of ML capable components on-die, such as CPU, GPU, and accelerators. These different components are capable of independently performing inference but with very different power-performance characteristics. In this article, we provide a quantitative evaluation of the inference capabilities of the different components on mobile SoCs. We also present insights behind their respective powerperformance behavior. Finally, we explore the performance limit of the mobile SoCs by synergistically engaging all the components concurrently. We observe that a mobile SoC provides up to 2x improvement with parallel inference when all its components are engaged, as opposed to engaging only one component.Keywords
All Related Versions
This publication has 5 references indexed in Scilit:
- High-Throughput CNN Inference on Embedded ARM Big.LITTLE Multicore ProcessorsIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2019
- Machine Learning at Facebook: Understanding Inference at the Edge2019
- Weighted Quantization-Regularization in DNNs for Weight Memory Minimization Toward HW ImplementationIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2018
- Deep Residual Learning for Image Recognition2016
- Going deeper with convolutions2015