Model Poisoning Attack on Neural Network Without Reference Data
- 25 May 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computers
- Vol. 72 (10) , 2978-2989
Abstract
Due to the substantial computational cost of neural network training, adopting third-party models has become increasingly popular. However, recent works demonstrate that third-party models can be poisoned. Nonetheless, most model poisoning attacks require reference data, e.g., training dataset or data belonging to the target label, making them difficult to launch in practice. In this paper, we propose a reference data independent model poisoning attack that can (1) directly search for sensitive features with respect to the target label, (2) quantify the positive and negative effects of the model parameters on sensitive features, and (3) accomplish the training of poisoned model by our parameter selective update strategy. The extensive evaluation on datasets with a few classes and numerous classes show that the attack is (I) effective: the trigger input can be labeled as a deliberate class by the poisoned model with high probability; (II) covert: the performance of the poisoned model is almost indistinguishable from the intact model on non-trigger inputs; and (III) straightforward: an adversary only needs a little background knowledge to launch the attack. Overall, the evaluation results show that our attack achieves $95\%$ , $100\%$ , $81\%$ , $96\%$ , and $96\%$ success rates on Cifar10, Cifar100, ISIC2018, FaceScrub, and ImageNet datasets, respectively.Keywords
Funding Information
- National Key Research and Development Program of China (2021YFB3100400)
- National Natural Science Foundation of China (62202276, 62232010, 62072278)
- Shandong Science Fund for Excellent Young Scholars (2022HWYQ-038)
This publication has 33 references indexed in Scilit:
- Detecting Adversarial Image Examples in Deep Neural Networks with Adaptive Noise ReductionIEEE Transactions on Dependable and Secure Computing, 2018
- Learning without ForgettingIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- Yes, Machine Learning Can Be More Secure! A Case Study on Android Malware DetectionIEEE Transactions on Dependable and Secure Computing, 2017
- Towards Evaluating the Robustness of Neural Networks2017
- Dermatologist-level classification of skin cancer with deep neural networksNature, 2017
- Learning Deep Features for Discriminative Localization2016
- The Limitations of Deep Learning in Adversarial Settings2016
- MADAM: Effective and Efficient Behavior-based Android Malware Detection and PreventionIEEE Transactions on Dependable and Secure Computing, 2016
- DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving2015
- Privacy-Preserving Deep Learning2015