Benchmarking the Robustness of Quantized Models
Preprint
Abstract
Quantization has emerged as an essential technique for deploying deep neural networks (DNNs) on devices with limited resources. However, quantized models exhibit vulnerabilities when exposed to various noises in real-world applications. Despite the importance of evaluating the impact of quantization on robustness, existing research on this topic is limited and often disregards established principles of robustness evaluation, resulting in incomplete and inconclusive findings. To address this gap, we thoroughly evaluated the robustness of quantized models against various noises (adversarial attacks, natural corruptions, and systematic noises) on ImageNet. Extensive experiments demonstrate that lower-bit quantization is more resilient to adversarial attacks but is more susceptible to natural corruptions and systematic noises. Notably, our investigation reveals that impulse noise (in natural corruptions) and the nearest neighbor interpolation (in systematic noises) have the most significant impact on quantized models. Our research contributes to advancing the robust quantization of models and their deployment in real-world scenarios.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- Discrete Cosine Transform Network for Guided Depth Map Super-Resolution2022
- Defensive Patches for Robust Recognition in the Physical World2022
- JointPruning: Pruning Networks Along Multiple Dimensions for Efficient Point Cloud ProcessingIEEE Transactions on Circuits and Systems for Video Technology, 2021
- Dual Attention Suppression Attack: Generate Adversarial Camouflage in Physical World2021
- Interpreting and Improving Adversarial Robustness of Deep Neural Networks With Neuron SensitivityIEEE Transactions on Image Processing, 2020
- Bias-Based Universal Adversarial Patch Attack for Automatic Check-Out2020
- Spatiotemporal Attacks for Embodied Agents2020
- Multi-Dimensional Pruning: A Unified Framework for Model Compression2020
- MobileNetV2: Inverted Residuals and Linear Bottlenecks2018
- Deep Residual Learning for Image Recognition2016