The repeatability of code defect classifications
Authors:
- I. WieczorekIWI. Wieczorek
- 1 January 1998
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 10719458,p. 322-333
Abstract
Counts of defects found during the various defect defection activities in software projects and their classification provide a basis for product quality evaluation and process improvement. However, since defect classifications are subjective, it is necessary to ensure that they are repeatable (i.e., that the classification is not dependent on the individual). We evaluate a slight adaptation of a commonly used defect classification scheme that has been applied in IBM's Orthogonal Defect Classification work, and in the SEI's Personal Software Process. The evaluation utilizes the Kappa statistic. We use defect data from code inspections conducted during a development project. Our results indicate that the classification scheme is in general repeatable. We further evaluate classes of defects to find out if confusion between some categories is more common, and suggest a potential improvement to the scheme.Keywords
This publication has 24 references indexed in Scilit:
- A promising approach to two-person software review in educational environmentJournal of Systems and Software, 1998
- Learning from our mistakes with defect causal analysisIEEE Software, 1998
- Using the GQM paradigm to investigate influential factors for software process improvementJournal of Systems and Software, 1997
- Assessing software designs using capture-recapture methodsIEEE Transactions on Software Engineering, 1993
- Interjudge Agreement and the Maximum Value of KappaEducational and Psychological Measurement, 1989
- Testing the Normal Approximation and Minimal Sample Size Requirements of Weighted Kappa When the Number of Categories is LargeApplied Psychological Measurement, 1981
- Measuring nominal scale agreement among many raters.Psychological Bulletin, 1971
- Large sample standard errors of kappa and weighted kappa.Psychological Bulletin, 1969
- Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit.Psychological Bulletin, 1968
- A Coefficient of Agreement for Nominal ScalesEducational and Psychological Measurement, 1960