Research Progress of Flaky Tests
Authors:
- 1 March 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 639-646
Abstract
A flaky test is a test that both passes and fails periodically without any code changes, and its uncontrolled uncertainty will destroy the value of the test suites and even cause developers to distrust the test results. Recently, researches in the flaky test have received broad attention in the software test community to reduce the manual maintenance cost of flaky tests by developers. In this survey, we conducted comprehensive research progress on the flaky test We identified 31 relevant studies and summarized the following aspects of the flaky test: root causes and factors, analyzing the impact, detecting and classifying techniques, and fixing approaches. This survey also identifies open research challenges to be further explored in future work.Keywords
Funding Information
- Ministry of Education
- Northwestern Polytechnical University
This publication has 38 references indexed in Scilit:
- Automatic repair of real bugs in java: a large-scale experiment on the defects4j datasetEmpirical Software Engineering, 2016
- An empirical study on the effects of code visibility on program testabilitySoftware Quality Journal, 2016
- Detecting Assumptions on Deterministic Implementations of Non-deterministic Specifications2016
- Efficient dependency detection for safe Java test acceleration2015
- Reliable testing: detecting state-polluting tests to prevent test dependency2015
- An empirical analysis of flaky tests2014
- Automated unit test generation for classes with environment dependencies2014
- Defects4J: a database of existing faults to enable controlled testing studies for Java programs2014
- Unit test virtualization with VMVM2014
- Automated testing of GUI applications: Models, tools, and controlling flakiness2013