Empirical Evaluation of Cross-Release Effort-Aware Defect Prediction Models
- 13 October 2016
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 214-221
Abstract
To prioritize quality assurance efforts, various fault prediction models have been proposed. However, the best performing fault prediction model is unknown due to three major drawbacks: (1) comparison of few fault prediction models considering small number of data sets, (2) use of evaluation measures that ignore testing efforts and (3) use of n-fold cross-validation instead of the more practical cross-release validation. To address these concerns, we conducted cross-release evaluation of 11 fault density prediction models using data sets collected from 2 releases of 25 open source software projects with an effort-aware performance measure known as Norm(Popt). Our result shows that, whilst M5 and K* had the best performances, they were greatly influenced by the percentage of faulty modules present and size of data set. Using Norm(Popt) produced an overall average performance of more than 50% across all the selected models clearly indicating the importance of considering testing efforts in building fault-prone prediction models.Keywords
This publication has 25 references indexed in Scilit:
- Software fault prediction metrics: A systematic literature reviewInformation and Software Technology, 2013
- Evaluating defect prediction approaches: a benchmark and an extensive comparisonEmpirical Software Engineering, 2011
- A systematic and comprehensive investigation of methods to build and evaluate fault prediction modelsJournal of Systems and Software, 2010
- Techniques for evaluating fault prediction modelsEmpirical Software Engineering, 2008
- Boosting Algorithms: Regularization, Prediction and Model FittingStatistical Science, 2007
- Comparative Assessment of Software Quality Classification Techniques: An Empirical Case StudyEmpirical Software Engineering, 2004
- Least angle regressionThe Annals of Statistics, 2004
- Predicting risk of software changesBell Labs Technical Journal, 2002
- Quantitative analysis of faults and failures in a complex software systemIEEE Transactions on Software Engineering, 2000
- Application of neural networks for predicting program faultsComputational Geosciences, 1995