Architectural Analysis of Deep Learning on Edge Accelerators
Authors:
- 22 September 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-7
Abstract
As computer architectures continue to integrate application-specific hardware, it is critical to understand the relative performance of devices for maximum app acceleration. The goal of benchmarking suites, such as MLPerf for analyzing machine-learning (ML) hardware performance, is to standardize a fair comparison of different hardware architectures. However, there are many apps that are not well represented by these standards that require different workloads, such as ML models and datasets, to achieve similar goals. Additionally, many devices feature hardware optimized for data types other than 32-bit floating-point numbers, the standard representation defined by MLPerf. Edge-computing devices often feature app-specific hardware to offload common operations found in ML apps from the constrained CPU. This research analyzes multiple low-power compute architectures that feature ML-specific hardware on a case study of handwritten Chinese character recognition. Specifically, AlexNet and a custom version of GoogLeNet are benchmarked in terms of their streaming latency for optical character recognition. Considering these models are custom and not the most widely used, many architectures are not specifically optimized for them. The performance of these models can stress devices in different, yet insightful, ways that generalizations of the performance of other models can be drawn from. The NVIDIA Jetson AGX Xavier (AGX), Intel Neural Compute Stick 2 (NCS2), and Google Edge TPU architectures are analyzed with respect to their performance. The design of the AGX and TPU devices showcased the lowest streaming latency for AlexNet and GoogLeNet, respectively. Additionally, the tightly-integrated N CS2 design showed the best generalizability in performance and efficiency across neural networks.Keywords
This publication has 14 references indexed in Scilit:
- Deep learning in the fogInternational Journal of Distributed Sensor Networks, 2019
- DeepHome2019
- Deep Learning for Breast Cancer Diagnosis from Mammograms—A Comparative StudyJournal of Imaging, 2019
- A Survey on Deep LearningACM Computing Surveys, 2018
- Squeezing Deep Learning into Mobile and Embedded DevicesIEEE Pervasive Computing, 2017
- An Early Resource Characterization of Deep Learning on Wearables, Smartphones and Internet-of-Things Devices2015
- High performance offline handwritten Chinese character recognition using GoogLeNet and directional feature maps2015
- Going deeper with convolutions2015
- Caffe2014
- Image processing with neural networks—a reviewPattern Recognition, 2002