Comparison of various methods for handling incomplete data in software engineering databases
- 10 December 2005
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 102-111
Abstract
Increasing the awareness of how missing data affects software predictive accuracy has led to increasing numbers of missing data techniques (MDTs). This paper investigates the robustness and accuracy of eight popular techniques for tolerating incomplete training and test data using tree-based models. MDTs were compared by artificially simulating different proportions, patterns, and mechanisms of missing data. A 4-way repeated measures design was employed to analyze the data. The simulation results suggest important differences. Listwise deletion is substantially inferior while multiple imputation (MI) represents a superior approach to handling missing data. Decision tree single imputation and surrogate variables splitting are more severely impacted by missing values distributed among all attributes. MI should be used if the data contain many missing values. If few values are missing, any of the MDTs might be considered. Choice of technique should be guided by pattern and mechanisms of missing data.Keywords
This publication has 20 references indexed in Scilit:
- An evaluation of k-nearest neighbour imputation using likert data2004
- Analyzing data sets with missing data: an empirical evaluation of imputation methods and likelihood-based methodsIEEE Transactions on Software Engineering, 2001
- Validating the ISO/IEC 15504 measures of software development process capabilityJournal of Systems and Software, 2000
- Imputation of Missing Data in Industrial DatabasesApplied Intelligence, 1999
- A critique of software defect prediction modelsIEEE Transactions on Software Engineering, 1999
- Multiple Imputation for Multivariate Missing-Data Problems: A Data Analyst's PerspectiveMultivariate Behavioral Research, 1998
- Analysis of Incomplete Multivariate Data1997
- Multiple Imputation After 18+ YearsJournal of the American Statistical Association, 1996
- Multiple Imputation for Nonresponse in Surveys1987
- Induction of decision treesMachine Learning, 1986