Poison Forensics: Traceback of Data Poisoning Attacks in Neural Networks
Preprint
Abstract
In adversarial machine learning, new defenses against attacks on deep learning systems are routinely broken soon after their release by more powerful attacks. In this context, forensic tools can offer a valuable complement to existing defenses, by tracing back a successful attack to its root cause, and offering a path forward for mitigation to prevent similar attacks in the future. In this paper, we describe our efforts in developing a forensic traceback tool for poison attacks on deep neural networks. We propose a novel iterative clustering and pruning solution that trims "innocent" training samples, until all that remains is the set of poisoned data responsible for the attack. Our method clusters training samples based on their impact on model parameters, then uses an efficient data unlearning method to prune innocent clusters. We empirically demonstrate the efficacy of our system on three types of dirty-label (backdoor) poison attacks and three types of clean-label poison attacks, across domains of computer vision and malware classification. Our system achieves over 98.4% precision and 96.8% recall across all attacks. We also show that our system is robust against four anti-forensics measures specifically designed to attack it.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Certified Robustness of Nearest Neighbors against Data Poisoning and Backdoor AttacksProceedings of the AAAI Conference on Artificial Intelligence, 2022
- Strong Data Augmentation Sanitizes Poisoning and Backdoor Attacks Without an Accuracy Tradeoff2021
- Intrinsic Certified Robustness of Bagging against Data Poisoning AttacksProceedings of the AAAI Conference on Artificial Intelligence, 2021
- ALchemist: Fusing Application and Audit Logs for Precise Attack Provenance without Instrumentation2021
- Gotta Catch'Em All: Using Honeypots to Catch Adversarial Attacks on Neural Networks2020
- Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation2020
- A Marauder's Map of Security and Privacy in Machine Learning2018
- Digital Forensics Explained2012
- Handbook of Digital Forensics and Investigation2010
- LIII. On lines and planes of closest fit to systems of points in spaceJournal of Computers in Education, 1901