Randomized Adversarial Training via Taylor Expansion
Authors:
- 1 June 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 16447-16457
Abstract
In recent years, there has been an explosion of research into developing more robust deep neural networks against adversarial examples. Adversarial training appears as one of the most successful methods. To deal with both the robustness against adversarial examples and the accuracy over clean examples, many works develop enhanced adversarial training methods to achieve various trade-offs between them [[19], [38], [80]], Leveraging over the studies [8], [32] that smoothed update on weights during training may help find flat minima and improve generalization, we suggest reconciling the robustness-accuracy trade-off from another perspective, i.e., by adding random noise into deterministic weights. The randomized weights enable our design of a novel adversarial training method via Taylor expansion of a small Gaussian noise, and we show that the new adversarial training method can flatten loss landscape and find flat minima. With PGD, CW, and Auto Attacks, an extensive set of experiments demonstrate that our method enhances the state-of-the-art adversarial training methods, boosting both robustness and clean accuracy. The code is available at https://github.com/Alexkael/Randomized-Adversarial-Training.Keywords
All Related Versions
This publication has 18 references indexed in Scilit:
- Entropy-SGD: biasing gradient descent into wide valleys*Journal of Statistical Mechanics: Theory and Experiment, 2019
- Distributionally Adversarial AttackProceedings of the AAAI Conference on Artificial Intelligence, 2019
- MobileNetV2: Inverted Residuals and Linear Bottlenecks2018
- Towards Evaluating the Robustness of Neural Networks2017
- Practical Black-Box Attacks against Machine Learning2017
- Deep Residual Learning for Image Recognition2016
- Rethinking the Inception Architecture for Computer Vision2016
- PAC-Bayesian model averaging1999
- Flat MinimaNeural Computation, 1997
- Modeling by shortest data descriptionAutomatica, 1978