Enabling ImageNet-Scale Deep Learning on MCUs for Accurate and Efficient Inference
- 9 November 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 11 (7) , 11471-11479
Abstract
Conventional approaches to TinyML achieve high accuracy by deploying the largest deep learning model with highest input resolutions that fit within the size constraints imposed by the microcontroller’s (MCUs) fast internal storage and memory. In this paper, we perform an in-depth analysis of prior works to show that models derived within these constraints suffer from low accuracy and, surprisingly, high latency. We propose an alternative approach that enables the deployment of efficient models with low inference latency, but free from the constraints of internal memory. We take a holistic view of typical MCU architectures, and utilise plentiful but slower external memories to relax internal storage and memory constraints. To avoid the lower speed of external memory impacting inference latency, we build on the TinyOps inference framework, which performs operation partitioning and uses overlays via DMA, to accelerate the latency. Using insights from our study, we deploy efficient models from the TinyOps design space onto a range of embedded MCUs achieving record performance on TinyML ImageNet classification with up to 6.7% higher accuracy and 1.4x faster latency compared to state-of-the-art internal memory approaches.Keywords
Funding Information
- U.K. Research and Innovation (UKRI) Centre for Doctoral Training in Machine Intelligence for Nano-Electronic Devices and Systems (EP/S024298/1)
- Engineering and Physical Sciences Research Council (EPSRC) International Centre for Spatial Computational Learning (EP/S030069/1)
- ARM Ltd., Cambridge, U.K.
This publication has 18 references indexed in Scilit:
- TinyOps: ImageNet Scale Deep Learning on Microcontrollers2022
- EEG signal analysis for epileptic seizures detection by applying Data Mining techniquesInternet of Things, 2021
- Searching for MobileNetV32019
- FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search2019
- MnasNet: Platform-Aware Neural Architecture Search for Mobile2019
- From Cloud Down to Things: An Overview of Machine Learning in Internet of ThingsIEEE Internet of Things Journal, 2019
- MobileNetV2: Inverted Residuals and Linear Bottlenecks2018
- Edge Computing for the Internet of Things: A Case StudyIEEE Internet of Things Journal, 2018
- Deep Residual Learning for Image Recognition2016
- Internet of Things for Smart CitiesIEEE Internet of Things Journal, 2014