Distribution-aware Adaptive Multi-bit Quantization
Authors:
- 1 June 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 9277-9286
Abstract
In this paper, we explore the compression of deep neural networks by quantizing the weights and activations into multi-bit binary networks (MBNs). A distribution-aware multi-bit quantization (DMBQ) method that incorporates the distribution prior into the optimization of quantization is proposed. Instead of solving the optimization in each iteration, DMBQ search the optimal quantization scheme over the distribution space beforehand, and select the quantization scheme during training using a fast lookup table based strategy. Based upon DMBQ, we further propose loss-guided bit-width allocation (LBA) to adaptively quantize and even prune the neural network. The first-order Taylor expansion is applied to build a metric for evaluating the loss sensitivity of the quantization of each channel, and automatically adjust the bit-width of weights and activations channel-wisely. We extend our method to image classification tasks and experimental results show that our method not only outperforms state-of-the-art quantized networks in terms of accuracy but also is more efficient in terms of training time compared with state-of-the-art MBNs, even for the extremely low bit width (below 1-bit) quantization cases.Keywords
Funding Information
- National Science Foundation
- Fundamental Research Funds for the Central Universities
This publication has 17 references indexed in Scilit:
- KCNN: Kernel-wise Quantization to Remarkably Decrease Multiplications in Convolutional Neural Network2019
- Fully Quantized Network for Object Detection2019
- Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference2018
- Deep Learning with Low Precision by Half-Wave Gaussian Quantization2017
- Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image DenoisingIEEE Transactions on Image Processing, 2017
- XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks2016
- Deep Residual Learning for Image Recognition2016
- Fully convolutional networks for semantic segmentation2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- New types of deep neural network learning for speech recognition and related applications: an overview2013