Backdoor attacks against learning systems
Authors:
- 1 October 2017
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-9
Abstract
Many of today's machine learning (ML) systems are composed by an array of primitive learning modules (PLMs). The heavy use of PLMs significantly simplifies and expedites the system development cycles. However, as most PLMs are contributed and maintained by third parties, their lack of standardization or regulation entails profound security implications. In this paper, for the first time, we demonstrate that potentially harmful PLMs incur immense threats to the security of ML-powered systems. We present a general class of backdoor attacks in which maliciously crafted PLMs trigger host systems to malfunction in a predictable manner once predefined conditions are present. We validate the feasibility of such attacks by empirically investigating a state-of-the-art skin cancer screening system. For example, it proves highly probable to force the system to misdiagnose a targeted victim, without any prior knowledge about how the system is built or trained. Further, we discuss the root causes behind the success of PLM-based attacks, which point to the characteristics of today's ML models: high dimensionality, non-linearity, and non-convexity. Therefore, the issue seems industry-wide.Keywords
This publication has 18 references indexed in Scilit:
- Dermatologist-level classification of skin cancer with deep neural networksNature, 2017
- Reliable Third-Party Library Detection in Android and its Security Applications2016
- Accessorize to a Crime2016
- Following Devil's Footprints: Cross-Platform Analysis of Potentially Harmful Libraries on Android and iOS2016
- Support vector machines under adversarial label contaminationNeurocomputing, 2015
- Deep learningNature, 2015
- Glove: Global Vectors for Word Representation2014
- Adversarial machine learning2011
- Can machine learning be secure?2006
- Approximation capabilities of multilayer feedforward networksNeural Networks, 1991