EMQ: Evolving Training-free Proxies for Automated Mixed Precision Quantization
Open Access
- 1 October 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 17030-17040
Abstract
Mixed-Precision Quantization (MQ) can achieve a competitive accuracy-complexity trade-off for models. Conventional training-based search methods require time-consuming candidate training to search optimized per-layer bit-width configurations in MQ. Recently, some training-free approaches have presented various MQ proxies and significantly improve search efficiency. However, the correlation between these proxies and quantization accuracy is poorly understood. To address the gap, we first build the MQ-Bench-101, which involves different bit configurations and quantization results. Then, we observe that the existing training-free proxies perform weak correlations on the MQ-Bench-101. To efficiently seek superior proxies, we develop an automatic search of proxies framework for MQ via evolving algorithms. In particular, we devise an elaborate search space involving the existing proxies and perform an evolution search to discover the best correlated MQ proxy. We proposed a diversity-prompting selection strategy and compatibility screening protocol to avoid premature convergence and improve search efficiency. In this way, our Evolving proxies for Mixed-precision Quantization (EMQ) framework allows the auto-generation of proxies without heavy tuning and expert knowledge. Extensive experiments on ImageNet with various ResNet and MobileNet families demonstrate that our EMQ obtains superior performance than state-of-the-art mixed-precision methods at a significantly reduced cost. The code will be released.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China
This publication has 22 references indexed in Scilit:
- Improving One-Shot NAS with Shrinking-and-Expanding SupernetPattern Recognition, 2021
- HMQ: Hardware Friendly Mixed Precision Quantization Block for CNNs2020
- Search What You Want: Barrier Panelty NAS for Mixed Precision Quantization2020
- Single Path One-Shot Neural Architecture Search with Uniform Sampling2020
- One Weight Bitwidth to Rule Them All2020
- LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks2018
- Value-Aware Quantization for Training and Inference of Neural Networks2018
- Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference2018
- Deep Residual Learning for Image Recognition2016
- Microsoft COCO: Common Objects in Context2014