DUBIOUS: Detecting Unknown Backdoored Input by Observing Unusual Signatures
Authors:
- 30 October 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 696-702
Abstract
Trojan / backdoor attacks represent a serious threat when using machine learning models that are either supplied by unreliable 3rd parties, or trained on data that contain deliberately poisoned samples. For such situations, a malicious actor can control outputs of the model on specially crafted inputs. It is not yet possible to accurately detect if a model was poisoned, so security considerations require augmenting Trojan detection methods with mitigation strategies. Mitigation measures should reduce the model’s vulnerability to poisoning attacks while maintaining its accuracy. Our goal is thus to determine if an input is triggering Trojan behavior, and if so, reject the model’s prediction for that input. We do this by first perturbing clean samples multiple times, and then gathering special statistics on the outputs of these perturbations. These statistics are referred to as a sample’s signature. The signature is built across multiple perturbation magnitudes to better capture the perturbations’ effects. Given a novel sample at test time, we build its signature in the same way, and compare it to those of the known clean examples. We can then apply outlier detection to determine if the input was indeed poisoned. In this paper, we examine Trojan attack mitigation using standard datasets in multiple modalities, namely image classification, natural language processing, and malware detection.Keywords
This publication has 13 references indexed in Scilit:
- Design and Evaluation of a Multi-Domain Trojan Detection Method on Deep Neural NetworksIEEE Transactions on Dependable and Secure Computing, 2021
- ONION: A Simple and Effective Defense Against Textual Backdoor Attacks2021
- RAP: Robustness-Aware Perturbations for Defending against Backdoor Attacks on NLP Models2021
- SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systems2020
- Generalization to Mitigate Synonym Substitution Attacks2020
- STRIP2019
- Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks2019
- BadNets: Evaluating Backdooring Attacks on Deep Neural NetworksIEEE Access, 2019
- Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks2018
- Malicious PDF detection using metadata and structural features2012