Implementation and Optimization of Neural Networks for Tiny Hardware Devices
Authors:
- 10 November 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 191-196
Abstract
Traditionally, neural network inferencing on tiny hardware devices took place in a centralized server-based manner. With more real-time applications coming into play, where security and latency are a concern, there has become a need to move inferencing to the edge. This paper describes a machine learning pipeline to carry neural networks from their initial forms to compressed forms deployable on tiny hardware devices, while maintaining acceptable accuracies of the optimized models. We will review the different software optimization techniques used to compress neural networks to their deployable forms. The prototype is a proof of concept showing that applying knowledge distillation from a highly accurate ResNet20 model to a simple CNN student model, followed by post-training quantization, achieves good multi-class accuracy on a constrained Arduino Nano 33 BLE Sense at low power consumption and with low inferencing latency.Keywords
This publication has 28 references indexed in Scilit:
- Democratization of AI, Albeit Constrained IoT Devices & Tiny ML, for Creating a Sustainable Food Future2020
- Ultra-Low Power Rectangular Field Programmable Analogue Arrays For Biomedical Applications2019
- Rate Splitting and Common Message Decoding for MIMO C-RAN Systems2019
- Neural Architecture Search2019
- Edge Computing in IoT-Based ManufacturingIEEE Communications Magazine, 2018
- Energy-Efficient Architecture for Wireless Sensor Networks in Healthcare ApplicationsIEEE Access, 2018
- A novel secure conference communication in IoT devices based on memristors2017
- Efficient Processing of Deep Neural Networks: A Tutorial and SurveyProceedings of the IEEE, 2017
- Towards the implementation of Multi‐band Multi‐standard Software‐Defined Radio using Dynamic Partial ReconfigurationInternational Journal of Communication Systems, 2017
- Introducing MicroBlaze as an infrastructure for performance modeling2011