Automated unit test generation for classes with environment dependencies
Authors:
Open Access
- 15 September 2014
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 79-90
Abstract
Automated test generation for object-oriented software typically consists of producing sequences of calls aiming at high code coverage. In practice, the success of this process may be inhibited when classes interact with their environment, such as the file system, network, user-interactions, etc. This leads to two major problems: First, code that depends on the environment can sometimes not be fully covered simply by generating sequences of calls to a class under test, for example when execution of a branch depends on the contents of a file. Second, even if code that is environment-dependent can be covered, the resulting tests may be unstable, i.e., they would pass when first generated, but then may fail when executed in a different environment. For example, tests on classes that make use of the system time may have failing assertions if the tests are executed at a different time than when they were generated. In this paper, we apply bytecode instrumentation to automatically separate code from its environmental dependencies, and extend the EvoSuite Java test generation tool such that it can explicitly set the state of the environment as part of the sequences of calls it generates. Using a prototype implementation, which handles a wide range of environmental interactions such as the file system, console inputs and many non-deterministic functions of the Java virtual machine (JVM), we performed experiments on 100 Java projects randomly selected from SourceForge (the SF100 corpus). The results show significantly improved code coverage - in some cases even in the order of +80%/+90%. Furthermore, our techniques reduce the number of unstable tests by more than 50%.Keywords
Funding Information
- Norges Forskningsråd
- Engineering and Physical Sciences Research Council (EP/K030353/1)
- Seventh Framework Programme (295261)
This publication has 14 references indexed in Scilit:
- Unit test virtualization with VMVM2014
- Improving search-based test suite generation with dynamic symbolic execution2013
- Whole Test Suite GenerationIEEE Transactions on Software Engineering, 2012
- Random Testing: Theoretical Results and Practical ImplicationsIEEE Transactions on Software Engineering, 2011
- Mutation-Driven Generation of Unit Tests and OraclesIEEE Transactions on Software Engineering, 2011
- EvoSuite2011
- An empirical study of testing file-system-dependent software with mock objects2009
- DART2005
- Mock object creation for test factoring2004
- JCrasher: an automatic robustness tester for JavaSoftware: Practice and Experience, 2004