Design and Analysis of a Neural Network Inference Engine Based on Adaptive Weight Compression
- 2 February 2018
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
- Vol. 38 (1) , 109-121
Abstract
Neural networks generally require significant memory capacity/bandwidth to store/access a large number of synaptic weights. This paper presents design of an energy-efficient neural network inference engine based on adaptive weight compression using a JPEG image encoding algorithm. To maximize compression ratio with minimum accuracy loss, the quality factor of the JPEG encoder is adaptively controlled depending on the accuracy impact of each block. With 1% accuracy loss, the proposed approach achieves 63.4× compression for multilayer perceptron (MLP) and 31.3× for LeNet-5 with the MNIST dataset, and 15.3× for AlexNet and 10.2× for ResNet-50 with ImageNet. The reduced memory requirement leads to higher throughput and lower energy for neural network inference (3× effective memory bandwidth and 22× lower system energy for MLP).Keywords
Funding Information
- Office of Naval Research
- National Science Foundation (CNS 1054429)
This publication has 26 references indexed in Scilit:
- 14.1 A 2.9TOPS/W deep convolutional neural network SoC in FD-SOI 28nm for intelligent embedded systems2017
- 14.2 DNPU: An 8.1TOPS/W reconfigurable CNN-RNN processor for general-purpose deep neural networks2017
- Deep Residual Learning for Image Recognition2016
- 14.5 Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks2016
- A power-aware digital feedforward neural network platform with backpropagation driven approximate synapses2015
- A Bayesian regularized artificial neural network for stock market forecastingExpert Systems with Applications, 2013
- Low-rank matrix factorization for Deep Neural Network training with high-dimensional output targets2013
- Improved JPEG compression algorithm based on saliency maps2012
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- Entropy and Image CompressionJournal of Visual Communication and Image Representation, 1993