Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
Authors:
- Jonathan UesatoJUJonathan Uesato
- Brendan O'DonoghueBOBrendan O'Donoghue
- Aaron van Den OordAOAaron van Den Oord
- Pushmeet KohliPKPushmeet Kohli
Preprint
Abstract
This paper investigates recently proposed approaches for defending against adversarial examples and evaluating adversarial robustness. We motivate 'adversarial risk' as an objective for achieving models robust to worst-case inputs. We then frame commonly used attacks and evaluation metrics as defining a tractable surrogate objective to the true adversarial risk. This suggests that models may optimize this surrogate rather than the true adversarial risk. We formalize this notion as 'obscurity to an adversary,' and develop tools and heuristics for identifying obscured models and designing transparent models. We demonstrate that this is a significant problem in practice by repurposing gradient-free optimization techniques into adversarial attacks, which we use to decrease the accuracy of several recently proposed defenses to near zero. Our hope is that our formulations and results will help researchers to develop more powerful defenses.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Adversarial Examples: Attacks and Defenses for Deep LearningIEEE Transactions on Neural Networks and Learning Systems, 2019
- Adversarial Examples in the Physical World2018
- Defense Against Adversarial Attacks Using High-Level Representation Guided Denoiser2018
- Feature VisualizationDistill, 2017
- Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks2017
- Adversarial Examples for Evaluating Reading Comprehension Systems2017
- Empirical risk minimization for heavy-tailed lossesThe Annals of Statistics, 2015
- Robust linear least squares regressionThe Annals of Statistics, 2011