Evaluating Random Mutant Selection at Class-Level in Projects with Non-Adequate Test Suites
Preprint
Abstract
Mutation testing is a standard technique to evaluate the quality of a test suite. Due to its computationally intensive nature, many approaches have been proposed to make this technique feasible in real case scenarios. Among these approaches, uniform random mutant selection has been demonstrated to be simple and promising. However, works on this area analyze mutant samples at project level mainly on projects with adequate test suites. In this paper, we fill this lack of empirical validation by analyzing random mutant selection at class level on projects with non-adequate test suites. First, we show that uniform random mutant selection underachieves the expected results. Then, we propose a new approach named weighted random mutant selection which generates more representative mutant samples. Finally, we show that representative mutant samples are larger for projects with high test adequacy.Keywords
All Related Versions
- Version 1, 2016-07-08, ArXiv
- Published version: , 1.
This publication has 9 references indexed in Scilit:
- Are mutants a valid substitute for real faults in software testing?2014
- A survey of software testing practices in CanadaJournal of Systems and Software, 2013
- Analysis of Ordinal Categorical Data2010
- A Survey of Coverage-Based Testing ToolsThe Computer Journal, 2007
- Toward the determination of sufficient mutant operators for C†Software Testing, Verification and Reliability, 2001
- Software unit test coverage and adequacyACM Computing Surveys, 1997
- An experimental determination of sufficient mutant operatorsACM Transactions on Software Engineering and Methodology, 1996
- Reducing the cost of mutation testing: An empirical studyJournal of Systems and Software, 1995
- Hints on Test Data Selection: Help for the Practicing ProgrammerComputer, 1978