High-Throughput CNN Inference on Embedded ARM Big.LITTLE Multicore Processors
Top Cited Papers
- 30 September 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
- Vol. 39 (10) , 2254-2267
Abstract
IoT Edge intelligence requires Convolutional Neural Network (CNN) inference to take place in the edge devices itself. ARM big.LITTLE architecture is at the heart of prevalent commercial edge devices. It comprises of single-ISA heterogeneous cores grouped into multiple homogeneous clusters that enable power and performance trade-offs. All cores are expected to be simultaneously employed in inference to attain maximal throughput. However, high communication overhead involved in parallelization of computations from convolution kernels across clusters is detrimental to throughput. We present an alternative framework called Pipe-it that employs pipelined design to split convolutional layers across clusters while limiting parallelization of their respective kernels to the assigned cluster. We develop a performance-prediction model that utilizes only the convolutional layer descriptors to predict the execution time of each layer individually on all permitted core configurations (type and count). Pipe-it then exploits the predictions to create a balanced pipeline using an efficient design space exploration algorithm. Pipe-it on average results in a 39% higher throughput than the highest antecedent throughput.Keywords
All Related Versions
Funding Information
- Singapore Ministry of Education Academic Research Fund Tier 2 (MOE2015-T2-2-088)
This publication has 18 references indexed in Scilit:
- Modeling the Resource Requirements of Convolutional Neural Networks on Mobile Devices2017
- Toward low-flying autonomous MAV trail navigation using deep neural networks for environmental awareness2017
- Deep Residual Learning for Image Recognition2016
- Quantized Convolutional Neural Networks for Mobile Devices2016
- ZNN -- A Fast and Scalable Algorithm for Training 3D Convolutional Networks on Multi-core and Many-Core Shared Memory Machines2016
- DeepX: A Software Accelerator for Low-Power Deep Learning Inference on Mobile Devices2016
- An Early Resource Characterization of Deep Learning on Wearables, Smartphones and Internet-of-Things Devices2015
- Going deeper with convolutions2015
- Hierarchical power management for asymmetric multi-core in dark silicon era2013
- Single-ISA heterogeneous multi-core architectures: the potential for processor power reduction2003