Effort-aware just-in-time defect prediction: simple unsupervised models could be better than supervised models
Authors:
Top Cited Papers
Open Access
- 1 November 2016
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 157-168
Abstract
Unsupervised models do not require the defect data to build the prediction models and hence incur a low building cost and gain a wide application range. Consequently, it would be more desirable for practitioners to apply unsupervised models in effort-aware just-in-time (JIT) defect prediction if they can predict defect-inducing changes well. However, little is currently known on their prediction effectiveness in this context. We aim to investigate the predictive power of simple unsupervised models in effort-aware JIT defect prediction, especially compared with the state-of-the-art supervised models in the recent literature. We first use the most commonly used change metrics to build simple unsupervised models. Then, we compare these unsupervised models with the state-of-the-art supervised models under cross-validation, time-wise-cross-validation, and across-project prediction settings to determine whether they are of practical value. The experimental results, from open-source software systems, show that many simple unsupervised models perform better than the state-of-the-art supervised models in effort-aware JIT defect prediction.Keywords
Funding Information
- program A for Outstanding PhD candidate of Nanjing University.
- Natural Science Foundation of Jiangsu Province (BK20130014)
- PolyU Grant (4-6934)
- National Natural Science Foundation of China (61432001, 91418202, 61272082, 61300051, 61321491, 61472178, and 91318301)
- Hong Kong Competitive Earmarked Research Grant (PolyU5219/06E)
- National Key Basic Research and Development Program of China (2014CB340702)
This publication has 45 references indexed in Scilit:
- Revisiting the Impact of Classification Techniques on the Performance of Defect Prediction Models2015
- Are Slice-Based Cohesion Metrics Actually Useful in Effort-Aware Post-Release Fault-Proneness Prediction? An Empirical StudyIEEE Transactions on Software Engineering, 2014
- An in-depth study of the potentially confounding effect of class size in fault predictionACM Transactions on Software Engineering and Methodology, 2014
- Time variance and defect prediction in software projectsEmpirical Software Engineering, 2011
- Testing the theory of relative defect proneness for closed-source softwareEmpirical Software Engineering, 2010
- Defect prediction from static code features: current results, limitations, new approachesAutomated Software Engineering, 2010
- A systematic and comprehensive investigation of methods to build and evaluate fault prediction modelsJournal of Systems and Software, 2010
- Theory of relative defect pronenessEmpirical Software Engineering, 2008
- Predicting risk of software changesBell Labs Technical Journal, 2002
- Predicting fault incidence using software change historyIEEE Transactions on Software Engineering, 2000