Rauf Izmailov
Rauf Izmailov has a total of 155 co-authors. Here are the top co-authors by number of shared publications:
S. Ganguly
16 shared publicationsB. Sengupta
8 shared publicationsSamrat Ganguly
NEC Laboratories America, Inc., USA
7 shared publications- Vladimir Vapnik
Columbia University
6 shared publications A. Iwata
Networking Res. Labs., NEC Corp., Kanagawa, Japan
5 shared publicationsS.K. Biswas
5 shared publications
Latest Publications
- 30 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 696-702
DUBIOUS: Detecting Unknown Backdoored Input by Observing Unusual Signatures
Trojan / backdoor attacks represent a serious threat when using machine learning models that are either supplied by unreliable 3rd parties, or trained on data that contain deliberately poisoned samples. For such situations, a malicious actor can control outputs of the model on specially crafted inputs. It is not yet possible to accurately detect if a model was poisoned, so security considerations require augmenting Trojan detection methods with mitigation strategies. Mitigation measures should reduce the model’s vulnerability to poisoning attacks while maintaining its accuracy. Our goal is thus to determine if an input is triggering Trojan behavior, and if so, reject the model’s prediction for that input. We do this by first perturbing clean samples multiple times, and then gathering special statistics on the outputs of these perturbations. These statistics are referred to as a sample’s signature. The signature is built across multiple perturbation magnitudes to better capture the perturbations’ effects. Given a novel sample at test time, we build its signature in the same way, and compare it to those of the known clean examples. We can then apply outlier detection to determine if the input was indeed poisoned. In this paper, we examine Trojan attack mitigation using standard datasets in multiple modalities, namely image classification, natural language processing, and malware detection.
- 30 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 696-702
DUBIOUS: Detecting Unknown Backdoored Input by Observing Unusual Signatures
Trojan / backdoor attacks represent a serious threat when using machine learning models that are either supplied by unreliable 3rd parties, or trained on data that contain deliberately poisoned samples. For such situations, a malicious actor can control outputs of the model on specially crafted inputs. It is not yet possible to accurately detect if a model was poisoned, so security considerations require augmenting Trojan detection methods with mitigation strategies. Mitigation measures should reduce the model’s vulnerability to poisoning attacks while maintaining its accuracy. Our goal is thus to determine if an input is triggering Trojan behavior, and if so, reject the model’s prediction for that input. We do this by first perturbing clean samples multiple times, and then gathering special statistics on the outputs of these perturbations. These statistics are referred to as a sample’s signature. The signature is built across multiple perturbation magnitudes to better capture the perturbations’ effects. Given a novel sample at test time, we build its signature in the same way, and compare it to those of the known clean examples. We can then apply outlier detection to determine if the input was indeed poisoned. In this paper, we examine Trojan attack mitigation using standard datasets in multiple modalities, namely image classification, natural language processing, and malware detection.
- 30 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 703-709
An Improved Nested Training Approach to Mitigate Clean-label Attacks against Malware Classifiers
Machine Learning (ML) models are being adopted as state of the art tools to defend systems against cybersecurity threats. Despite their high accuracy, such models remain vulnerable to various types of adversarial attacks. In cybersecurity, ML models are frequently trained on crowd-sourced datasets, which naturally allows attackers to introduce backdoor samples into malware detectors by using the recently proposed class of clean-label poisoning attacks. The presence of such backdoor samples in training datasets can later be used by attackers to modify a trained model’s behavior. Nested Training is an ensemble-based method for detecting poisoned samples by leveraging disagreements within the outputs of a diverse set of ensemble models on training data points. In the paper, we adapt the Nested Training approach to various cyber domains such as Android APKs and PDF Files, in which the distributions of data and the effects of poisoning have quite different properties. We then compare the resulting improved Nested Training approach with existing methods and demonstrate the ability of the approach to mitigate the effects of clean-label poisoned attacks and to recover an original Machine Learning model’s accuracy.
- 30 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 703-709
An Improved Nested Training Approach to Mitigate Clean-label Attacks against Malware Classifiers
Machine Learning (ML) models are being adopted as state of the art tools to defend systems against cybersecurity threats. Despite their high accuracy, such models remain vulnerable to various types of adversarial attacks. In cybersecurity, ML models are frequently trained on crowd-sourced datasets, which naturally allows attackers to introduce backdoor samples into malware detectors by using the recently proposed class of clean-label poisoning attacks. The presence of such backdoor samples in training datasets can later be used by attackers to modify a trained model’s behavior. Nested Training is an ensemble-based method for detecting poisoned samples by leveraging disagreements within the outputs of a diverse set of ensemble models on training data points. In the paper, we adapt the Nested Training approach to various cyber domains such as Android APKs and PDF Files, in which the distributions of data and the effects of poisoning have quite different properties. We then compare the resulting improved Nested Training approach with existing methods and demonstrate the ability of the approach to mitigate the effects of clean-label poisoned attacks and to recover an original Machine Learning model’s accuracy.
- Open Access
Minimax Defense against Gradient-based Adversarial Attacks
State-of-the-art adversarial attacks are aimed at neural network classifiers. By default, neural networks use gradient descent to minimize their loss function. The gradient of a classifier's loss function is used by gradient-based adversarial attacks to generate adversarially perturbed images. We pose the question whether another type of optimization could give neural network classifiers an edge. Here, we introduce a novel approach that uses minimax optimization to foil gradient-based adversarial attacks. Our minimax classifier is the discriminator of a generative adversarial network (GAN) that plays a minimax game with the GAN generator. In addition, our GAN generator projects all points onto a manifold that is different from the original manifold since the original manifold might be the cause of adversarial attacks. To measure the performance of our minimax defense, we use adversarial attacks - Carlini Wagner (CW), DeepFool, Fast Gradient Sign Method (FGSM) - on three datasets: MNIST, CIFAR-10 and German Traffic Sign (TRAFFIC). Against CW attacks, our minimax defense achieves 98.07% (MNIST-default 98.93%), 73.90% (CIFAR-10-default 83.14%) and 94.54% (TRAFFIC-default 96.97%). Against DeepFool attacks, our minimax defense achieves 98.87% (MNIST), 76.61% (CIFAR-10) and 94.57% (TRAFFIC). Against FGSM attacks, we achieve 97.01% (MNIST), 76.79% (CIFAR-10) and 81.41% (TRAFFIC). Our Minimax adversarial approach presents a significant shift in defense strategy for neural network classifiers.
Open AccessMinimax Defense against Gradient-based Adversarial Attacks
State-of-the-art adversarial attacks are aimed at neural network classifiers. By default, neural networks use gradient descent to minimize their loss function. The gradient of a classifier's loss function is used by gradient-based adversarial attacks to generate adversarially perturbed images. We pose the question whether another type of optimization could give neural network classifiers an edge. Here, we introduce a novel approach that uses minimax optimization to foil gradient-based adversarial attacks. Our minimax classifier is the discriminator of a generative adversarial network (GAN) that plays a minimax game with the GAN generator. In addition, our GAN generator projects all points onto a manifold that is different from the original manifold since the original manifold might be the cause of adversarial attacks. To measure the performance of our minimax defense, we use adversarial attacks - Carlini Wagner (CW), DeepFool, Fast Gradient Sign Method (FGSM) - on three datasets: MNIST, CIFAR-10 and German Traffic Sign (TRAFFIC). Against CW attacks, our minimax defense achieves 98.07% (MNIST-default 98.93%), 73.90% (CIFAR-10-default 83.14%) and 94.54% (TRAFFIC-default 96.97%). Against DeepFool attacks, our minimax defense achieves 98.87% (MNIST), 76.61% (CIFAR-10) and 94.57% (TRAFFIC). Against FGSM attacks, we achieve 97.01% (MNIST), 76.79% (CIFAR-10) and 81.41% (TRAFFIC). Our Minimax adversarial approach presents a significant shift in defense strategy for neural network classifiers.
- Open Access
Privacy Leakage Avoidance with Switching Ensembles
We consider membership inference attacks, one of the main privacy issues in machine learning. These recently developed attacks have been proven successful in determining, with confidence better than a random guess, whether a given sample belongs to the dataset on which the attacked machine learning model was trained. Several approaches have been developed to mitigate this privacy leakage but the tradeoff performance implications of these defensive mechanisms (i.e., accuracy and utility of the defended machine learning model) are not well studied yet. We propose a novel approach of privacy leakage avoidance with switching ensembles (PASE), which both protects against current membership inference attacks and does that with very small accuracy penalty, while requiring acceptable increase in training and inference time. We test our PASE method, along with the the current state-of-the-art PATE approach, on three calibration image datasets and analyze their tradeoffs.
Open AccessPrivacy Leakage Avoidance with Switching Ensembles
We consider membership inference attacks, one of the main privacy issues in machine learning. These recently developed attacks have been proven successful in determining, with confidence better than a random guess, whether a given sample belongs to the dataset on which the attacked machine learning model was trained. Several approaches have been developed to mitigate this privacy leakage but the tradeoff performance implications of these defensive mechanisms (i.e., accuracy and utility of the defended machine learning model) are not well studied yet. We propose a novel approach of privacy leakage avoidance with switching ensembles (PASE), which both protects against current membership inference attacks and does that with very small accuracy penalty, while requiring acceptable increase in training and inference time. We test our PASE method, along with the the current state-of-the-art PATE approach, on three calibration image datasets and analyze their tradeoffs.
- Open Access
Membership Model Inversion Attacks for Deep Networks
With the increasing adoption of AI, inherent security and privacy vulnerabilities formachine learning systems are being discovered. One such vulnerability makes itpossible for an adversary to obtain private information about the types of instancesused to train the targeted machine learning model. This so-called model inversionattack is based on sequential leveraging of classification scores towards obtaininghigh confidence representations for various classes. However, for deep networks,such procedures usually lead to unrecognizable representations that are uselessfor the adversary. In this paper, we introduce a more realistic definition of modelinversion, where the adversary is aware of the general purpose of the attackedmodel (for instance, whether it is an OCR system or a facial recognition system),and the goal is to find realistic class representations within the corresponding lower-dimensional manifold (of, respectively, general symbols or general faces). To thatend, we leverage properties of generative adversarial networks for constructinga connected lower-dimensional manifold, and demonstrate the efficiency of ourmodel inversion attack that is carried out within that manifold.
Open AccessMembership Model Inversion Attacks for Deep Networks
With the increasing adoption of AI, inherent security and privacy vulnerabilities formachine learning systems are being discovered. One such vulnerability makes itpossible for an adversary to obtain private information about the types of instancesused to train the targeted machine learning model. This so-called model inversionattack is based on sequential leveraging of classification scores towards obtaininghigh confidence representations for various classes. However, for deep networks,such procedures usually lead to unrecognizable representations that are uselessfor the adversary. In this paper, we introduce a more realistic definition of modelinversion, where the adversary is aware of the general purpose of the attackedmodel (for instance, whether it is an OCR system or a facial recognition system),and the goal is to find realistic class representations within the corresponding lower-dimensional manifold (of, respectively, general symbols or general faces). To thatend, we leverage properties of generative adversarial networks for constructinga connected lower-dimensional manifold, and demonstrate the efficiency of ourmodel inversion attack that is carried out within that manifold.
- Open Access
- 18 July 2018
- Published by Springer Nature in Machine Learning
- Vol. 108 (3) , 381-423
Rethinking statistical learning theory: learning using statistical invariants
- 18 July 2018
- Published by Springer Nature in Machine Learning
- Vol. 108 (3) , 381-423
Open AccessRethinking statistical learning theory: learning using statistical invariants
- Open Access
Detection under Privileged Information
For well over a quarter century, detection systems have been driven by models learned from input features collected from real or simulated environments. An artifact (e.g., network event, potential malware sample, suspicious email) is deemed malicious or non-malicious based on its similarity to the learned model at runtime. However, the training of the models has been historically limited to only those features available at runtime. In this paper, we consider an alternate learning approach that trains models using "privileged" information--features available at training time but not at runtime--to improve the accuracy and resilience of detection systems. In particular, we adapt and extend recent advances in knowledge transfer, model influence, and distillation to enable the use of forensic or other data unavailable at runtime in a range of security domains. An empirical evaluation shows that privileged information increases precision and recall over a system with no privileged information: we observe up to 7.7% relative decrease in detection error for fast-flux bot detection, 8.6% for malware traffic detection, 7.3% for malware classification, and 16.9% for face recognition. We explore the limitations and applications of different privileged information techniques in detection systems. Such techniques provide a new means for detection systems to learn from data that would otherwise not be available at runtime.
Open AccessDetection under Privileged Information
For well over a quarter century, detection systems have been driven by models learned from input features collected from real or simulated environments. An artifact (e.g., network event, potential malware sample, suspicious email) is deemed malicious or non-malicious based on its similarity to the learned model at runtime. However, the training of the models has been historically limited to only those features available at runtime. In this paper, we consider an alternate learning approach that trains models using "privileged" information--features available at training time but not at runtime--to improve the accuracy and resilience of detection systems. In particular, we adapt and extend recent advances in knowledge transfer, model influence, and distillation to enable the use of forensic or other data unavailable at runtime in a range of security domains. An empirical evaluation shows that privileged information increases precision and recall over a system with no privileged information: we observe up to 7.7% relative decrease in detection error for fast-flux bot detection, 8.6% for malware traffic detection, 7.3% for malware classification, and 16.9% for face recognition. We explore the limitations and applications of different privileged information techniques in detection systems. Such techniques provide a new means for detection systems to learn from data that would otherwise not be available at runtime.
- 1 July 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-8
Application of Learning Using Privileged Information(LUPI): Botnet Detection
Traffic anomaly detection is primarily concerned with identifying malicious traffic patterns in a much larger stream of benign traffic. Traditionally, this is achieved by selecting a very specialized set of traffic-based features that are used for both training a model, as well as for detection at runtime. This paper introduces a novel method of anomaly detection that breaks the assumption that the same set of features should be used for both training and runtime detection. Building upon the concept of Learning Using Privileged Information (LUPI), this paper shows how to build a detection model that uses an extended set of features, some that are available only during training, and some also available at runtime, with results significantly superior to the traditional learning methods. We demonstrate the usage of the privileged features in detecting malicious botnet activities in a network, activities shown to be challenging to detect in past works.
- 1 July 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-8
Application of Learning Using Privileged Information(LUPI): Botnet Detection
Traffic anomaly detection is primarily concerned with identifying malicious traffic patterns in a much larger stream of benign traffic. Traditionally, this is achieved by selecting a very specialized set of traffic-based features that are used for both training a model, as well as for detection at runtime. This paper introduces a novel method of anomaly detection that breaks the assumption that the same set of features should be used for both training and runtime detection. Building upon the concept of Learning Using Privileged Information (LUPI), this paper shows how to build a detection model that uses an extended set of features, some that are available only during training, and some also available at runtime, with results significantly superior to the traditional learning methods. We demonstrate the usage of the privileged features in detecting malicious botnet activities in a network, activities shown to be challenging to detect in past works.
- Open Access
- 24 March 2017
- Published by Association for Computing Machinery (ACM)
- p. 73-80
Feature Cultivation in Privileged Information-augmented Detection
Modern detection systems use sensor outputs available in the deployment environment to probabilistically identify attacks. These systems are trained on past or synthetic feature vectors to create a model of anomalous or normal behavior. Thereafter, run-time collected sensor outputs are compared to the model to identify attacks (or the lack of attack). While this approach to detection has been proven to be effective in many environments, it is limited to training on only features that can be reliably collected at detection time. Hence, they fail to leverage the often vast amount of ancillary information available from past forensic analysis and post-mortem data. In short, detection systems do not train (and thus do not learn from) features that are unavailable or too costly to collect at run-time. Recent work proposed an alternate model construction approach that integrates forensic "privilege" information---features reliably available at training time, but not at run-time---to improve accuracy and resilience of detection systems. In this paper, we further evaluate two of proposed techniques to model training with privileged information: knowledge transfer, and model influence. We explore the cultivation of privileged features, the efficiency of those processes and their influence on the detection accuracy. We observe that the improved integration of privileged features makes the resulting detection models more accurate. Our evaluation shows that use of privileged information leads to up to 8.2% relative decrease in detection error for fast-flux bot detection over a system with no privileged information, and 5.5% for malware classification.
- 24 March 2017
- Published by Association for Computing Machinery (ACM)
- p. 73-80
Open AccessFeature Cultivation in Privileged Information-augmented Detection
Modern detection systems use sensor outputs available in the deployment environment to probabilistically identify attacks. These systems are trained on past or synthetic feature vectors to create a model of anomalous or normal behavior. Thereafter, run-time collected sensor outputs are compared to the model to identify attacks (or the lack of attack). While this approach to detection has been proven to be effective in many environments, it is limited to training on only features that can be reliably collected at detection time. Hence, they fail to leverage the often vast amount of ancillary information available from past forensic analysis and post-mortem data. In short, detection systems do not train (and thus do not learn from) features that are unavailable or too costly to collect at run-time. Recent work proposed an alternate model construction approach that integrates forensic "privilege" information---features reliably available at training time, but not at run-time---to improve accuracy and resilience of detection systems. In this paper, we further evaluate two of proposed techniques to model training with privileged information: knowledge transfer, and model influence. We explore the cultivation of privileged features, the efficiency of those processes and their influence on the detection accuracy. We observe that the improved integration of privileged features makes the resulting detection models more accurate. Our evaluation shows that use of privileged information leads to up to 8.2% relative decrease in detection error for fast-flux bot detection over a system with no privileged information, and 5.5% for malware classification.
- 20 February 2017
- Published by Springer Nature in Annals of Mathematics and Artificial Intelligence
- Vol. 81 (1-2) , 3-19
Knowledge transfer in SVM and neural networks
- 20 February 2017
- Published by Springer Nature in Annals of Mathematics and Artificial Intelligence
- Vol. 81 (1-2) , 3-19
Knowledge transfer in SVM and neural networks