GRT: Program-Analysis-Guided Random Testing (T)
Authors:
- Lei MaLMLei Ma
- Cyrille ArthoCACyrille Artho
- Cheng ZhangCZCheng Zhang
- Hiroyuki SatoHSHiroyuki Sato
- Johannes GmeinerJGJohannes Gmeiner
- Rudolf RamlerRRRudolf Ramler
- 1 November 2015
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 212-223
Abstract
We propose Guided Random Testing (GRT), which uses static and dynamic analysis to include information on program types, data, and dependencies in various stages of automated test generation. Static analysis extracts knowledge from the system under test. Test coverage is further improved through state fuzzing and continuous coverage analysis. We evaluated GRT on 32 real-world projects and found that GRT outperforms major peer techniques in terms of code coverage (by 13 %) and mutation score (by 9 %). On the four studied benchmarks of Defects4J, which contain 224 real faults, GRT also shows better fault detection capability than peer techniques, finding 147 faults (66 %). Furthermore, in an in-depth evaluation on the latest versions of ten popular real-world projects, GRT successfully detects over 20 unknown defects that were confirmed by developers.Keywords
This publication has 34 references indexed in Scilit:
- Reim & ReImInferACM SIGPLAN Notices, 2012
- A survey of new trends in symbolic execution for software testing and analysisInternational Journal on Software Tools for Technology Transfer, 2009
- Pex–White Box Test Generation for .NET2008
- CUTEACM SIGSOFT Software Engineering Notes, 2005
- DARTACM SIGPLAN Notices, 2005
- Purity and Side Effect Analysis for Java Programs2005
- Symstra: A Framework for Generating Object-Oriented Unit Tests Using Symbolic Execution2005
- Test input generation with java PathFinderACM SIGSOFT Software Engineering Notes, 2004
- JCrasher: an automatic robustness tester for JavaSoftware: Practice and Experience, 2004
- Proportional sampling strategy: a compendium and some insightsJournal of Systems and Software, 2001