Dealing with Class Noise in Large Training Datasets for Malware Detection
Authors:
- 1 September 2011
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 401-407
Abstract
This paper presents the ways we explored until now for detecting and dealing with the class noise found in large annotated datasets used for training the classifiers that we have previously designed for industrial-scale malware identification. First we established a number of distance-based filtering rules that allow us to identify different "levels'' of potential noise in the training data, and secondly we analysed the effects produced by either removal or "cleaning'' of the potentially-noised records on the performances of our simplest classifiers. We show that a careful distance-based filtering can lead to sensibly better results in malware detection.Keywords
This publication has 10 references indexed in Scilit:
- Malware detection using machine learning2009
- Detection of malicious code by applying machine learning classifiers on static features: A state-of-the-art surveyInformation Security Technical Report, 2009
- Behavioral detection of malware: from a survey towards an established taxonomyJournal of Computer Virology and Hacking Techniques, 2008
- Learning and Classification of Malware Behavior2008
- Limits of Static Analysis for Malware Detection2007
- Bridging Local and Global Data Cleansing: Identifying Class Noise in Large, Distributed Data DatasetsData Mining and Knowledge Discovery, 2006
- Improving automated land cover mapping by identifying and eliminating mislabeled observations from training data2002
- An Introduction to Support Vector Machines and Other Kernel-based Learning Methods2000
- Large Margin Classification Using the Perceptron AlgorithmMachine Learning, 1999
- Support-vector networksMachine Learning, 1995