Dealing with noise in defect prediction
Authors:
Open Access
- 21 May 2011
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 481-490
Abstract
Many software defect prediction models have been built using historical defect data obtained by mining software repositories (MSR). Recent studies have discovered that data so collected contain noises because current defect collection practices are based on optional bug fix keywords or bug report links in change logs. Automatically collected defect data based on the change logs could include noises. This paper proposes approaches to deal with the noise in defect data. First, we measure the impact of noise on defect prediction models and provide guidelines for acceptable noise level. We measure noise resistant ability of two well-known defect prediction algorithms and find that in general, for large defect datasets, adding FP (false positive) or FN (false negative) noises alone does not lead to substantial performance differences. However, the prediction performance decreases significantly when the dataset contains 20%-35% of both FP and FN noises. Second, we propose a noise detection and elimination algorithm to address this problem. Our empirical study shows that our algorithm can identify noisy instances with reasonable accuracy. In addition, after eliminating the noises using our algorithm, defect prediction accuracy is improved. © 2011 ACMKeywords
This publication has 27 references indexed in Scilit:
- Predicting defects using network analysis on dependency graphs2008
- A comparative analysis of the efficiency of change metrics and static code attributes for defect prediction2008
- Missing Data in Software Engineering2008
- Learning from bug-introducing changes to prevent fault prone code2007
- Mining metrics to predict component failures2006
- Class Noise vs. Attribute Noise: A Quantitative StudyArtificial Intelligence Review, 2004
- Open-Source Change LogsEmpirical Software Engineering, 2004
- Analyzing data sets with missing data: an empirical evaluation of imputation methods and likelihood-based methodsIEEE Transactions on Software Engineering, 2001
- Software cost estimation with incomplete dataIEEE Transactions on Software Engineering, 2001
- A metrics suite for object oriented designIEEE Transactions on Software Engineering, 1994