Characterizing and Repairing Obsolete Android GUI Tests under UI Evolution
Open Access
- 1 October 2026
- journal article
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Software Engineering
- Vol. 3 (ISSTA) , 4789-4810
Abstract
Graphical user interface (GUI) tests are widely used in regression testing of mobile applications (apps) to validate app behavior from the user's perspective. However, frequent app evolution, such as UI redesigns and feature updates, often renders existing GUI tests obsolete, even when underlying functionality remains unchanged. Automatically repairing such tests is critical for maintaining test suites and reducing substantial manual effort. Despite the practical importance, there is still a lack of a systematic understanding of the characteristics of obsolete GUI tests and a publicly available benchmark to support their study. To fill this gap, we construct a benchmark comprising 736 obsolete GUI tests collected from 36 real-world mobile apps across 668 historical versions. We then conduct a large-scale empirical study that reveals two major challenges in repairing obsolete GUI tests: First, identifying the intended target widget is difficult because widget attributes are frequently missing or unstable, and visually similar widgets may correspond to different functionalities. Second, the target widget is often no longer directly reachable from the failure state, as it may be hidden behind additional UI interactions or relocated to another page. To address these challenges, we further propose GUIRevive, an automated GUI test repair approach that addresses semantic ambiguity in widget identification and target unreachability under UI evolution by integrating semantic-aware reasoning, functionality-preserving validation, and goal-guided UI exploration. Our evaluation shows that GUIRevive successfully repairs 86.4% of obsolete GUI tests and significantly outperforms state-of-the-art repair techniques by up to 220%. Moreover, GUIRevive has been deployed in industrial settings, achieving a 93% repair success rate on industrial mobile apps.Keywords
Funding Information
- National Research Foundation (NCRP25-P04-TAICeN)
- Cyber Security Agency of Singapore (NCRP25-P04-TAICeN)
This publication has 43 references indexed in Scilit:
- Combining Language and App Ui Analysis for the Automated Assessment of Bug Reproduction Steps2025
- Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs2024
- CogAgent: A Visual Language Model for GUI Agents2024
- An Empirical Analysis of UI-Based Flaky Tests2021
- How do Developers Test Android Applications?2017
- DroidBot: a lightweight UI-Guided test input generator for android2017
- ATOM: Automatic Maintenance of GUI Test Scripts for Evolving Mobile Applications2017
- Bug characteristics in open source softwareEmpirical Software Engineering, 2013
- Repairing GUI Test Suites Using a Genetic Algorithm2010
- Maintaining and evolving GUI-directed test scripts2009