QANS: Toward Quantized Neural Network Adversarial Noise Suppression
- 8 June 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
- Vol. 42 (12) , 4858-4870
Abstract
Neural network quantization techniques play an important role in efficiently deploying deep learning models on the hardware with limited computing and storage resources. Numerous applications of this technology, such as autopilot, necessitate not just efficiency, but also robustness. Research on the robustness of quantized networks against adversarial attacks is becoming one of the major points of interest. In this work, we rethink the impact of quantization on adversarial attacks and explore the boundary of the robustness of quantized neural networks. This study reveals that activation quantization can be used as a defense to weaken adversarial noise, but the robustness of quantized models is still limited by the amplification effect of network errors, including quantization errors and adversarial noise. To address this problem, we propose the Quantization Adversarial Noise Suppression (QANS) method that employs a Gaussian kernel regularization constraint to stabilize the model by restricting the perturbation error within two levels of tolerance. Extensive experiments are conducted with Wide ResNet and VGG-16 models on CIFAR-10 and SVHN datasets under different attack methods, including several white-box and black-box attacks. Experimental results show the proposed method achieves superior robustness to prior works.Keywords
Funding Information
- Shaanxi Key Technology Research and Development Program (2021ZDLGY07-01)
- National Natural Science Foundation of China (61972302, 61962019)
- Fundamental Research Funds for the Central Universities (20199236571)
This publication has 25 references indexed in Scilit:
- Understanding adversarial training: Increasing local stability of supervised models through robust optimizationNeurocomputing, 2018
- Weighted Quantization-Regularization in DNNs for Weight Memory Minimization Toward HW ImplementationIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2018
- Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference2018
- Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks2018
- ZOO2017
- Why Does Deep and Cheap Learning Work So Well?Journal of Statistical Physics, 2017
- Singular-Value-Decomposition Analysis of Associative Memory in a Neural NetworkJournal of the Physical Society of Japan, 2017
- Deep learningNature, 2015
- Recognition with local features: the kernel recipe2003
- A Global Geometric Framework for Nonlinear Dimensionality ReductionScience, 2000