Automatic generation of oracles for exceptional behaviors
Authors:
Open Access
- 18 July 2016
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 213-224
Abstract
Test suites should test exceptional behavior to detect faults in error-handling code. However, manually-written test suites tend to neglect exceptional behavior. Automatically-generated test suites, on the other hand, lack test oracles that verify whether runtime exceptions are the expected behavior of the code under test. This paper proposes a technique that automatically creates test oracles for exceptional behaviors from Javadoc comments. The technique uses a combination of natural language processing and run-time instrumentation. Our implementation, Toradocu, can be combined with a test input generation tool. Our experimental evaluation shows that Toradocu improves the fault-finding effectiveness of EvoSuite and Randoop test suites by 8% and 16% respectively, and reduces EvoSuite’s false positives by 33%.Keywords
This publication has 47 references indexed in Scilit:
- Eclat: Automatic Generation and Classification of Test Inputs2005
- Finding and preventing run-time error handling mistakesACM SIGPLAN Notices, 2004
- JCrasher: an automatic robustness tester for JavaSoftware: Practice and Experience, 2004
- Automatically checking an implementation against its formal specificationIEEE Transactions on Software Engineering, 2000
- A practical approach to programming with assertionsIEEE Transactions on Software Engineering, 1995
- Compilation of Z specifications into C for automatic test result evaluationLecture Notes in Computer Science, 1995
- The ASTOOT approach to testing object-oriented programsACM Transactions on Software Engineering and Methodology, 1994
- Test selection based on finite state modelsIEEE Transactions on Software Engineering, 1991
- An integrated verification and testing environmentSoftware: Practice and Experience, 1983
- Data Abstraction, Implementation, Specification, and TestingACM Transactions on Programming Languages and Systems, 1981