Performance Analysis of Maximal Risk Evaluation Formulas for Spectrum-Based Fault Localization
Open Access
- 5 January 2020
- journal article
- research article
- Published by MDPI AG in Applied Sciences
- Vol. 10 (1) , 398
Abstract
The effectiveness analysis of risk evaluation formulas has become a significant research area in spectrum-based fault localization (SBFL). The risk evaluation formula is designed and widely used to evaluate the likelihood of a program spectrum to be faulty. There are numerous empirical and theoretical studies to investigate and compare the performance between sixty risk evaluation formulas. According to previous research, these sixty risk evaluation formulas together form a partially ordered set. Among them, nine formulas are maximal. These nine formulas can further be grouped into five maximal risk evaluation formula groups so that formulas in the same group have the same performance. Moreover, previous research showed that we cannot theoretically compare formulas across these five maximal formula groups. However, experimental data “suggests” that a maximal formula in one group could outperform another one (from a different group) more frequently, though not always. This inspired us to further investigate the performance between any two maximal formulas in different maximal formula groups. Our approach involves two major steps. First, we propose a new condition to compare between two different maximal formulas. Based on this new condition, we present five different scenarios under which a formula performs better than another. This is different from the condition suggested by the previous theoretical study. We performed an empirical study to compare different maximal formulas using our condition. Our results showed that among two maximal risk evaluation formulas, it is feasible to identify one that can outperform the others more frequently.Keywords
Funding Information
- National Natural Science Foundation of China (61772423)
This publication has 29 references indexed in Scilit:
- A Survey on Software Fault LocalizationIEEE Transactions on Software Engineering, 2016
- A Revisit of a Theoretical Analysis on Spectrum-Based Fault Localization2015
- Slice-based statistical fault localizationJournal of Systems and Software, 2014
- Fault localization prioritizationACM Transactions on Software Engineering and Methodology, 2013
- Metamorphic slice: An application in spectrum-based fault localizationInformation and Software Technology, 2013
- A model for spectra-based software diagnosisACM Transactions on Software Engineering and Methodology, 2011
- A family of code coverage-based heuristics for effective fault localizationJournal of Systems and Software, 2010
- A practical evaluation of spectrum-based fault localizationJournal of Systems and Software, 2009
- Effective program debugging based on execution slices and inter-block data dependencyJournal of Systems and Software, 2006
- Smart debugging software architectural design in SDLJournal of Systems and Software, 2004