Mixed-Precision Quantization: Make the Best Use of Bits Where They Matter Most
Authors:
- Yiming FangYFYiming Fang
- Li ChenLCLi Chen
- Yunfei ChenYunfei Chen
- Weidong WangWWWeidong Wang
- Changsheng YouChangsheng You
Preprint
Abstract
Mixed-precision quantization offers superior performance to fixed-precision quantization. It has been widely used in signal processing, communication systems, and machine learning. In mixed-precision quantization, bit allocation is essential. Hence, in this paper, we propose a new bit allocation framework for mixed-precision quantization from a search perspective. First, we formulate a general bit allocation problem for mixed-precision quantization. Then we introduce the penalized particle swarm optimization (PPSO) algorithm to address the integer consumption constraint. To improve efficiency and avoid iterations on infeasible solutions within the PPSO algorithm, a greedy criterion particle swarm optimization (GC-PSO) algorithm is proposed. The corresponding convergence analysis is derived based on dynamical system theory. Furthermore, we apply the above framework to some specific classic fields, i.e., finite impulse response (FIR) filters, receivers, and gradient descent. Numerical examples in each application underscore the superiority of the proposed framework to the existing algorithms.Keywords
All Related Versions
This publication has 32 references indexed in Scilit:
- A Tutorial on Extremely Large-Scale MIMO for 6G: Fundamentals, Signal Processing, and ApplicationsIEEE Communications Surveys & Tutorials, 2024
- Mixed precision algorithms in numerical linear algebraActa Numerica, 2022
- Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Network2018
- Performance Analysis of Mixed-ADC Massive MIMO Systems Over Rician Fading ChannelsIEEE Journal on Selected Areas in Communications, 2017
- Mixed-ADC Massive MIMOIEEE Journal on Selected Areas in Communications, 2016
- Survey of Large-Scale MIMO SystemsIEEE Communications Surveys & Tutorials, 2015
- Telescoping rounding for suboptimal finite wordlength FIR digital filter designDigital Signal Processing, 2005
- The particle swarm optimization algorithm: convergence analysis and parameter selectionInformation Processing Letters, 2003
- The particle swarm - explosion, stability, and convergence in a multidimensional complex spaceIEEE Transactions on Evolutionary Computation, 2002
- Analog-to-digital converter survey and analysisIEEE Journal on Selected Areas in Communications, 1999