Auto-tuning Neural Network Quantization Framework for Collaborative Inference Between the Cloud and Edge
Preprint
Abstract
Recently, deep neural networks (DNNs) have been widely applied in mobile intelligent applications. The inference for the DNNs is usually performed in the cloud. However, it leads to a large overhead of transmitting data via wireless network. In this paper, we demonstrate the advantages of the cloud-edge collaborative inference with quantization. By analyzing the characteristics of layers in DNNs, an auto-tuning neural network quantization framework for collaborative inference is proposed. We study the effectiveness of mixed-precision collaborative inference of state-of-the-art DNNs by using ImageNet dataset. The experimental results show that our framework can generate reasonable network partitions and reduce the storage on mobile devices with trivial loss of accuracy.Keywords
All Related Versions
This publication has 4 references indexed in Scilit:
- Quantized CNN: A Unified Approach to Accelerate and Compress Convolutional NetworksIEEE Transactions on Neural Networks and Learning Systems, 2018
- Recent advances in efficient computation of deep convolutional neural networksFrontiers of Information Technology & Electronic Engineering, 2018
- Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research GroupsIEEE Signal Processing Magazine, 2012
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998