Testing Web Enabled Simulation at Scale Using Metamorphic Testing
Authors:
- 1 May 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 140-149
Abstract
We report on Facebook’s deployment of MIA (Meta-morphic Interaction Automaton). MIA is used to test Facebook’s Web Enabled Simulation, built on a web infrastructure of hundreds of millions of lines of code. MIA tackles the twin problems of test flakiness and the unknowable oracle problem. It uses metamorphic testing to automate continuous integration and regression test execution. MIA also plays the role of a test bot, automatically commenting on all relevant changes submitted for code review. It currently uses a suite of over 40 metamorphic test cases. Even at this extreme scale, a non-trivial metamorphic test suite subset yields outcomes within 20 minutes (sufficient for continuous integration and review processes). Furthermore, our offline mode simulation reduces test flakiness from approximately 50% (of all online tests) to 0% (offline). Metamorphic testing has been widely-studied for 22 years. This paper is the first reported deployment into an industrial continuous integration system.Keywords
This publication has 53 references indexed in Scilit:
- Search-based software engineeringACM Computing Surveys, 2012
- Automated metamorphic testing on the analyses of feature modelsInformation and Software Technology, 2011
- A survey of combinatorial testingACM Computing Surveys, 2011
- Automated functional testing of online search servicesSoftware Testing, Verification and Reliability, 2010
- Testability transformationIEEE Transactions on Software Engineering, 2004
- Simulation in the supply chain context: a surveyComputers in Industry, 2004
- Search-based software engineeringInformation and Software Technology, 2001
- Birds of a Feather: Homophily in Social NetworksAnnual Review of Sociology, 2001
- Software testability: the new verificationIEEE Software, 1995
- On Testing Non-Testable ProgramsThe Computer Journal, 1982