Abstract
Trojan / backdoor attacks represent a serious threat when using machine learning models that are either supplied by unreliable 3rd parties, or trained on data that contain deliberately poisoned samples. For such situations, a malicious actor can control outputs of the model on specially crafted inputs. It is not yet possible to accurately detect if a model was poisoned, so security considerations require augmenting Trojan detection methods with mitigation strategies. Mitigation measures should reduce the model’s vulnerability to poisoning attacks while maintaining its accuracy. Our goal is thus to determine if an input is triggering Trojan behavior, and if so, reject the model’s prediction for that input. We do this by first perturbing clean samples multiple times, and then gathering special statistics on the outputs of these perturbations. These statistics are referred to as a sample’s signature. The signature is built across multiple perturbation magnitudes to better capture the perturbations’ effects. Given a novel sample at test time, we build its signature in the same way, and compare it to those of the known clean examples. We can then apply outlier detection to determine if the input was indeed poisoned. In this paper, we examine Trojan attack mitigation using standard datasets in multiple modalities, namely image classification, natural language processing, and malware detection.