Neural Network Inference on Mobile SoCs
Authors:
- Siqi WangSWSiqi Wang
- Anuj PathaniaAPAnuj Pathania
- Tulika MitraTMTulika Mitra
Preprint
Abstract
The ever-increasing demand from mobile Machine Learning (ML) applications calls for evermore powerful on-chip computing resources. Mobile devices are empowered with heterogeneous multi-processor Systems-on-Chips (SoCs) to process ML workloads such as Convolutional Neural Network (CNN) inference. Mobile SoCs house several different types of ML capable components on-die, such as CPU, GPU, and accelerators. These different components are capable of independently performing inference but with very different power-performance characteristics. In this article, we provide a quantitative evaluation of the inference capabilities of the different components on mobile SoCs. We also present insights behind their respective power-performance behavior. Finally, we explore the performance limit of the mobile SoCs by synergistically engaging all the components concurrently. We observe that a mobile SoC provides up to 2x improvement with parallel inference when all its components are engaged, as opposed to engaging only one component.Keywords
All Related Versions
This publication has 3 references indexed in Scilit:
- High-Throughput CNN Inference on Embedded ARM Big.LITTLE Multicore ProcessorsIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2019
- Weighted Quantization-Regularization in DNNs for Weight Memory Minimization Toward HW ImplementationIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2018
- Roofline: An Insightful Visual Performance Model for Floating-Point Programs and Multicore Architectures2009