TestSage: Regression Test Selection for Large-Scale Web Service Testing
Authors:
- 1 April 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 430-440
Abstract
Regression testing is an important but expensive activity in software development. Among various types of tests, web service tests are usually one of the most expensive (due to network communications) but widely adopted types of tests in commercial software development. Regression test selection (RTS) aims to reduce the number of tests which need to be retested by only running tests that are affected by code changes. Although a large number of RTS techniques have been proposed in the past few decades, these techniques have not been adopted on large-scale web service testing. This is because most existing RTS techniques either require direct code dependency between tests and code under test or cannot be applied on large scale systems with enough efficiency. In this paper, we present a novel RTS technique, TestSage, that performs RTS for web service tests on large scale commercial software. With a small overhead, TestSage is able to collect fine grained (function level) dependency between test and service under test that do not directly depend on each other. TestSage has also been successfully applied to large complex systems with over a million functions. We conducted experiments of TestSage on a large scale backend service at Google. Experimental results show that TestSage reduces 34% of testing time when running all AEC (Analysis, Execution and Collection) phases, 50% of testing time while running without collection phase. TestSage has been integrated with internal testing framework at Google and runs day-to-day at the company.Keywords
This publication has 23 references indexed in Scilit:
- Practical regression test selection with dynamic file dependencies2015
- Large-scale cluster management at Google with Borg2015
- FaultTracer: a spectrum‐based approach to localizing failure‐inducing program editsJournal of Software: Evolution and Process, 2013
- Regression testing minimization, selection and prioritization: a surveySoftware Testing, Verification and Reliability, 2012
- A Qualitative Survey of Regression Testing Practices2010
- Regression Testing Web Services-based Applications2006
- ChiantiACM SIGPLAN Notices, 2004
- Regression test selection for Java software2001
- Change impact analysis for object-oriented programs2001
- A safe, efficient regression test selection techniqueACM Transactions on Software Engineering and Methodology, 1997