Cause reduction: delta debugging, even without bugs
- 31 March 2015
- journal article
- research article
- Published by Wiley in Software Testing, Verification and Reliability
- Vol. 26 (1) , 40-68
Abstract
Summary: What is a test casefor? Sometimes, to expose a fault. Tests can also exercise code, use memory or time, or produce desired output. Given a desired effect, a test case can be seen as acause, and its components divided into essential (required for effect) and accidental. Delta debugging is used for removing accidents from failing test cases, producing smaller test cases that are easier to understand. This paper extends delta debugging by simplifying test cases with respect toarbitrary effects, a generalization calledcause reduction. Suites produced by cause reduction provide effectivequick testsfor real‐world programs. For Mozilla's JavaScript engine, the reduced suite is possiblymoreeffective for finding faults. The effectiveness of reduction‐based suites persists through changes to the software, improving coverage by over 500 branches for versions up to 4 months later. Cause reduction has other applications, including improving seeded symbolic execution, where using reduced tests can often double the number of additional branches explored. Copyright © 2015 John Wiley & Sons, Ltd.Keywords
Funding Information
- National Science Foundation (CCF-1217824)
This publication has 60 references indexed in Scilit:
- An approach for experimentally evaluating effectiveness and efficiency of coverage criteria for software testingInternational Journal on Software Tools for Technology Transfer, 2008
- Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential ImpactEmpirical Software Engineering, 2005
- Empirical studies of test‐suite reductionSoftware Testing, Verification and Reliability, 2002
- Isolating failure-inducing thread schedulesACM SIGSOFT Software Engineering Notes, 2002
- Prioritizing test cases for regression testingIEEE Transactions on Software Engineering, 2001
- A comparative study of coarse- and fine-grained safe regression test-selection techniquesACM Transactions on Software Engineering and Methodology, 2001
- QuickCheckACM SIGPLAN Notices, 2000
- Simplifying failure-inducing inputACM SIGSOFT Software Engineering Notes, 2000
- Dividing strategies for the optimization of a test suiteInformation Processing Letters, 1996
- Dynamic program slicingACM SIGPLAN Notices, 1990