Improving the Robustness of Model Compression by On-Manifold Adversarial Training
Open Access
- 25 November 2021
- journal article
- research article
- Published by MDPI AG in Future Internet
- Vol. 13 (12) , 300
Abstract
Despite the advance in deep learning technology, assuring the robustness of deep neural networks (DNNs) is challenging and necessary in safety-critical environments, including automobiles, IoT devices in smart factories, and medical devices, to name a few. Furthermore, recent developments allow us to compress DNNs to reduce the size and computational requirements of DNNs to fit them into small embedded devices. However, how robust a compressed DNN can be has not been well studied in addressing its relationship to other critical factors, such as prediction performance and model sizes. In particular, existing studies on robust model compression have been focused on the robustness against off-manifold adversarial perturbation, which does not explain how a DNN will behave against perturbations that follow the same probability distribution as the training data. This aspect is relevant for on-device AI models, which are more likely to experience perturbations due to noise from the regular data observation environment compared with off-manifold perturbations provided by an external attacker. Therefore, this paper investigates the robustness of compressed deep neural networks, focusing on the relationship between the model sizes and the prediction performance on noisy perturbations. Our experiment shows that on-manifold adversarial training can be effective in building robust classifiers, especially when the model compression rate is high.Keywords
Funding Information
- Institute for Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No.2017-0-00142)
This publication has 23 references indexed in Scilit:
- Towards Evaluating the Robustness of Neural Networks2017
- Practical Black-Box Attacks against Machine Learning2017
- DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks2016
- Deep Residual Learning for Image Recognition2016
- Fast ConvNets Using Group-Wise Brain Damage2016
- Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks2016
- The Limitations of Deep Learning in Adversarial Settings2016
- CoSaMPCommunications of the ACM, 2010
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- Parameter estimation in non-Gaussian noiseGeophysical Journal International, 1988