Subpopulation Data Poisoning Attacks
Authors:
Open Access
- 12 November 2021
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 3104-3122
Abstract
Machine learning systems are deployed in critical settings, but they might fail in unexpected ways, impacting the accuracy of their predictions. Poisoning attacks against machine learning induce adversarial modification of data used by a machine learning algorithm to selectively change its output when it is deployed. In this work, we introduce a novel data poisoning attack called a subpopulation attack, which is particularly relevant when datasets are large and diverse. We design a modular framework for subpopulation attacks, instantiate it with different building blocks, and show that the attacks are effective for a variety of datasets and machine learning models. We further optimize the attacks in continuous domains using influence functions and gradient optimization methods. Compared to existing backdoor poisoning attacks, subpopulation attacks have the advantage of inducing misclassification in naturally distributed data points at inference time, making the attacks extremely stealthy. We also show that our attack strategy can be used to improve upon existing targeted attacks. We prove that, under some assumptions, subpopulation attacks are impossible to defend against, and empirically demonstrate the limitations of existing defenses against our attacks, highlighting the difficulty of protecting machine learning against this threat.Keywords
All Related Versions
Funding Information
- ARL (W911NF-13-2-0045)
This publication has 31 references indexed in Scilit:
- Age Progression/Regression by Conditional Adversarial Autoencoder2017
- Aggregated Residual Transformations for Deep Neural Networks2017
- Towards Evaluating the Robustness of Neural Networks2017
- Membership Inference Attacks Against Machine Learning Models2017
- Practical Black-Box Attacks against Machine Learning2017
- Automatically Evading Classifiers: A Case Study on PDF Malware Classifiers2016
- Poisoning behavioral malware clustering2014
- Evasion Attacks against Machine Learning at Test Time2013
- Fairness through awareness2012
- A Survey on Transfer LearningIEEE Transactions on Knowledge and Data Engineering, 2009