ZoomIn: Discovering Failures by Detecting Wrong Assertions
Authors:
- Fabrizio PastoreFPFabrizio Pastore
- Leonardo MarianiLMLeonardo Mariani
- 1 May 2015
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 1, 66-76
Abstract
Automatic testing, although useful, is still quite ineffective against faults that do not cause crashes or uncaught exceptions. In the majority of the cases automatic tests do not include oracles, and only in some cases they incorporate assertions that encode the observed behavior instead of the intended behavior, that is if the application under test produces a wrong result, the synthesized assertions will encode wrong expectations that match the actual behavior of the application. In this paper we present Zoom In, a technique that extends the fault-revealing capability of test case generation techniques from crash-only faults to faults that require non-trivial oracles to be detected. Zoom In exploits the knowledge encoded in the manual tests written by developers and the similarity between executions to automatically determine an extremely small set of suspicious assertions that are likely wrong and thus worth manual inspection. Early empirical results show that Zoom In has been able to detect 50% of the analyzed non-crashing faults in the Apache Commons Math library requiring the inspection of less than 1.5% of the assertions automatically generated by EvoSuite.Keywords
This publication has 28 references indexed in Scilit:
- Cross-checking oracles from intrinsic software redundancy2014
- Automated oracles: an empirical study on cost and effectiveness2013
- Understanding user understanding: determining correctness of generated program invariants2012
- EvoSuite2011
- Automatic generation of software behavioral models2008
- DSD-CrasherACM Transactions on Software Engineering and Methodology, 2008
- Pex–White Box Test Generation for .NET2008
- Mining object behavior with ADABU2006
- Semantic anomaly detection in online data sources2002
- Dynamically discovering likely program invariants to support program evolutionIEEE Transactions on Software Engineering, 2001