Adversarial Finetuning with Latent Representation Constraint to Mitigate Accuracy-Robustness Tradeoff
Authors:
- 1 October 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 4367-4378
Abstract
This paper addresses the tradeoff between standard accuracy on clean examples and robustness against adversarial examples in deep neural networks (DNNs). Although adversarial training (AT) improves robustness, it degrades the standard accuracy, thus yielding the tradeoff. To mitigate this tradeoff, we propose a novel AT method called ARREST, which comprises three components: (i) adversarial finetuning (AFT), (ii) representation-guided knowledge distillation (RGKD), and (iii) noisy replay (NR). AFT trains a DNN on adversarial examples by initializing its parameters with a DNN that is standardly pretrained on clean examples. RGKD and NR respectively entail a regularization term and an algorithm to preserve latent representations of clean examples during AFT. RGKD penalizes the distance between the representations of the standardly pretrained and AFT DNNs. NR switches input adversarial examples to nonadversarial ones when the representation changes significantly during AFT. By combining these components, ARREST achieves both high standard accuracy and robustness. Experimental results demonstrate that ARREST mitigates the tradeoff more effectively than previous AT-based methods do.Keywords
All Related Versions
This publication has 29 references indexed in Scilit:
- Transfer Without Forgetting2022
- Square Attack: A Query-Efficient Black-Box Adversarial Attack via Random Search2020
- Robustness via Curvature Regularization, and Vice Versa2019
- Disentangling Adversarial Robustness and Generalization2019
- YOLO9000: Better, Faster, Stronger2017
- DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks2016
- Deep Residual Learning for Image Recognition2016
- Going deeper with convolutions2015
- Fully convolutional networks for semantic segmentation2015
- Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem1989