AROID: Improving Adversarial Robustness Through Online Instance-Wise Data Augmentation
Open Access
- 24 August 2024
- journal article
- research article
- Published by Springer Nature in International Journal of Computer Vision
- Vol. 133 (2) , 929-950
Abstract
Deep neural networks are vulnerable to adversarial examples. Adversarial training (AT) is an effective defense against adversarial examples. However, AT is prone to overfitting which degrades robustness substantially. Recently, data augmentation (DA) was shown to be effective in mitigating robust overfitting if appropriately designed and optimized for AT. This work proposes a new method to automatically learn online, instance-wise, DA policies to improve robust generalization for AT. This is the first automated DA method specific for robustness. A novel policy learning objective, consisting of Vulnerability, Affinity and Diversity, is proposed and shown to be sufficiently effective and efficient to be practical for automatic DA generation during AT. Importantly, our method dramatically reduces the cost of policy search from the 5000 h of AutoAugment and the 412 h of IDBH to 9 h, making automated DA more practical to use for adversarial robustness. This allows our method to efficiently explore a large search space for a more effective DA policy and evolve the policy as training progresses. Empirically, our method is shown to outperform all competitive DA methods across various model architectures and datasets. Our DA policy reinforced vanilla AT to surpass several state-of-the-art AT methods regarding both accuracy and robustness. It can also be combined with those advanced AT methods to further boost robustness. Code and pre-trained models are available at: https://github.com/TreeLLi/AROID.Keywords
Funding Information
- China Scholarship Council
This publication has 26 references indexed in Scilit:
- Exploring misclassifications of robust neural networks to enhance adversarial attacksApplied Intelligence, 2023
- Mitigating robust overfitting via self-residual-calibration regularizationArtificial Intelligence, 2023
- Understanding and combating robust overfitting via input loss landscape analysis and regularizationPattern Recognition, 2022
- Faster AutoAugment: Learning Augmentation Strategies Using Backpropagation2020
- Differentiable Automatic Data Augmentation2020
- AutoAugment: Learning Augmentation Strategies From Data2019
- Identity Mappings in Deep Residual Networks2016
- Deep Residual Learning for Image Recognition2016
- ImageNet: A large-scale hierarchical image database2009
- Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning1992