The Impact of Feature Selection on Defect Prediction Performance: An Empirical Comparison
Authors:
- Zhou XuZXZhou Xu
- Jin LiuJLJin Liu
- Zijiang YangZYZijiang Yang
- Gege AnGAGege An
- Xiangyang JiaXJXiangyang Jia
- 1 October 2016
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 309-320
Abstract
Software defect prediction aims to determine whether a software module is defect-prone by constructing prediction models. The performance of such models is susceptible to the high dimensionality of the datasets that may include irrelevant and redundant features. Feature selection is applied to alleviate this issue. Because many feature selection methods have been proposed, there is an imperative need to analyze and compare these methods. Prior empirical studies may have potential controversies and limitations, such as the contradictory results, usage of private datasets and inappropriate statistical test techniques. This observation leads us to conduct a careful empirical study to reinforce the confidence of the experimental conclusions by considering several potential source of bias, such as the noise in the dataset and the dataset types. In this paper, we investigate the impact of 32 feature selection methods on the defect prediction performance over two versions of the NASA dataset (i.e., the noisy and clean NASA datasets) and one open source AEEEM dataset. We use a state-of-the-art double Scott-Knott test technique to analyze these methods. Experimental results show that the effectiveness of these feature selection methods on defect prediction performance varies significantly over all the datasets.Keywords
This publication has 55 references indexed in Scilit:
- Detecting Novel Associations in Large Data SetsScience, 2011
- Choosing software metrics for defect prediction: an investigation on feature selection techniquesSoftware: Practice and Experience, 2011
- A novel hybrid feature selection via Symmetrical Uncertainty ranking based local memetic search algorithmKnowledge-Based Systems, 2010
- Defect prediction from static code features: current results, limitations, new approachesAutomated Software Engineering, 2010
- Predicting defect-prone software modules using support vector machinesJournal of Systems and Software, 2008
- An introduction to ROC analysisPattern Recognition Letters, 2006
- A feature selection technique for classificatory analysisPattern Recognition Letters, 2005
- Consistency-based search in feature selectionArtificial Intelligence, 2003
- Comparing software prediction techniques using simulationIEEE Transactions on Software Engineering, 2001
- A Cluster Analysis Method for Grouping Means in the Analysis of Variance1974