Random or Genetic Algorithm Search for Object-Oriented Test Suite Generation?
Open Access
- 11 July 2015
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1367-1374
Abstract
Achieving high structural coverage is an important aim in software testing. Several search-based techniques have proved successful at automatically generating tests that achieve high coverage. However, despite the well- established arguments behind using evolutionary search algorithms (e.g., genetic algorithms) in preference to random search, it remains an open question whether the benefits can actually be observed in practice when generating unit test suites for object-oriented classes. In this paper, we report an empirical study on the effects of using a genetic algorithm (GA) to generate test suites over generating test suites incrementally with random search, by applying the EvoSuite unit test suite generator to 1,000 classes randomly selected from the SF110 corpus of open source projects. Surprisingly, the results show little difference between the coverage achieved by test suites generated with evolutionary search compared to those generated using random search. A detailed analysis reveals that the genetic algorithm covers more branches of the type where standard fitness functions provide guidance. In practice, however, we observed that the vast majority of branches in the analyzed projects provide no such guidance.Keywords
Funding Information
- Engineering and Physical Sciences Research Council (EP/K030353/1)
This publication has 26 references indexed in Scilit:
- Quantifying the Characteristics of Java Programs That May Influence Symbolic Execution from a Test Data Generation Perspective2014
- Handling test length bloatSoftware Testing, Verification and Reliability, 2013
- The Seed is Strong: Seeding Strategies in Search-Based Software Testing2012
- Whole Test Suite GenerationIEEE Transactions on Software Engineering, 2012
- Automated web application testing using search based software engineering2011
- EvoSuite2011
- TestFul: An Evolutionary Test Approach for Java2010
- Nighthawk2007
- JCrasher: an automatic robustness tester for JavaSoftware: Practice and Experience, 2004
- Evolutionary Testing of Flag Conditions2003