Are Slice-Based Cohesion Metrics Actually Useful in Effort-Aware Post-Release Fault-Proneness Prediction? An Empirical Study
- 12 November 2014
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Software Engineering
- Vol. 41 (4) , 331-357
Abstract
Background. Slice-based cohesion metrics leverage program slices with respect to the output variables of a module to quantify the strength of functional relatedness of the elements within the module. Although slice-based cohesion metrics have been proposed for many years, few empirical studies have been conducted to examine their actual usefulness in predicting fault-proneness. Objective. We aim to provide an in-depth understanding of the ability of slice-based cohesion metrics in effort-aware post-release fault-proneness prediction, i.e. their effectiveness in helping practitioners find post-release faults when taking into account the effort needed to test or inspect the code. Method. We use the most commonly used code and process metrics, including size, structural complexity, Halstead's software science, and code churn metrics, as the baseline metrics. First, we employ principal component analysis to analyze the relationships between slice-based cohesion metrics and the baseline metrics. Then, we use univariate prediction models to investigate the correlations between slice-based cohesion metrics and post-release fault-proneness. Finally, we build multivariate prediction models to examine the effectiveness of slice-based cohesion metrics in effort-aware post-release fault-proneness prediction when used alone or used together with the baseline code and process metrics. Results. Based on open-source software systems, our results show that: 1) slice-based cohesion metrics are not redundant with respect to the baseline code and process metrics; 2) most slice-based cohesion metrics are significantly negatively related to post-release fault-proneness; 3) slice-based cohesion metrics in general do not outperform the baseline metrics when predicting post-release fault-proneness; and 4) when used with the baseline metrics together, however, slice-based cohesion metrics can produce a statistically significant and practically important improvement of the effectiveness in effort-aware post-release fault-proneness prediction. Conclusion. Slice-based cohesion metrics are complementary to the most commonly used code and process metrics and are of practical value in the context of effort-aware post-release fault-proneness prediction.Keywords
Funding Information
- National Key Basic Research and Development Program of China (2014CB340702)
- National Natural Science Foundation of China (61432001, 91318301, 61321491, 61379045, 61272082, 61300051, 61472175)
- National Natural Science Foundation of Jiangsu Province (BK20130014)
- National Science and Technology Major Project of China (2012ZX01039-004)
- Hong Kong Competitive Earmarked Research (PolyU5219/06E)
- PolyU (4-6934)
This publication has 42 references indexed in Scilit:
- On the ability of complexity metrics to predict fault-prone classes in object-oriented systemsJournal of Systems and Software, 2010
- A systematic and comprehensive investigation of methods to build and evaluate fault prediction modelsJournal of Systems and Software, 2010
- Software similarity-based functional cohesion metricIET Software, 2009
- The effectiveness of software metrics in identifying error-prone classes in post-release software evolution processJournal of Systems and Software, 2008
- A controlled experiment for evaluating quality guidelines on the maintainability of object-oriented designsIEEE Transactions on Software Engineering, 2001
- Reexamining the fault density component size connectionIEEE Software, 1997
- Measuring functional cohesionIEEE Transactions on Software Engineering, 1994
- The detection of fault-prone programsIEEE Transactions on Software Engineering, 1992
- The program dependence graph in a software development environmentACM SIGPLAN Notices, 1984
- Multicollinearity in Regression Analysis: The Problem RevisitedReview of Economics and Statistics, 1967