Hawkeye: Change-targeted Testing for Android Apps based on Deep Reinforcement Learning
Open Access
- 14 April 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 298-308
Abstract
Android Apps are frequently updated to keep up with changing user, hardware, and business demands. Ensuring the correctness of App updates through extensive testing is crucial to avoid potential bugs reaching the end user. Existing Android testing tools generate GUI events that focus on improving the test coverage of the entire App rather than prioritising updates and impacted elements. Recent research has proposed change-focused testing but relies on random exploration to exercise change-impacted GUI elements that is ineffective and slow for large complex Apps with a huge input exploration space. At ByteDance, our established model-based GUI testing tool, Fastbot2, has been in successful deployment for nearly three years. Fastbot2 leverages event-activity transition models derived from past explorations to achieve enhanced test coverage efficiently. A pivotal insight we gained is that the knowledge of event-activity transitions is equally valuable in effectively targeting changes introduced by updates. This insight propelled our proposal for directed testing of updates with Hawkeye. Hawkeye excels in prioritizing GUI actions associated with code changes through deep reinforcement learning from historical exploration data. In our empirical evaluation, we rigorously compared Hawkeye with state-of-the-art tools like Fastbot2 and ARES on 10 popular open-source Apps and a commercial App. The results showcased that Hawkeye consistently outperforms Fastbot2 and ARES in generating GUI event sequences that effectively target changed functions, both in open-source and commercial App contexts. In real-world industrial deployment, Hawkeye is seamlessly integrated into our development pipeline, performing smoke testing for merge requests in a complex commercial App. The positive feedback received from our App development teams further affirmed Hawkeye's ability in testing App updates effectively.Keywords
All Related Versions
This publication has 26 references indexed in Scilit:
- Practical GUI Testing of Android Applications Via Model Abstraction and Refinement2019
- EHBDroid: Beyond GUI testing for Android applications2017
- Guided, stochastic model-based GUI testing of Android apps2017
- Automated model-based Android GUI testing using multi-level GUI comparison criteria2016
- Regression test selection for Android applications2016
- DroidMate2016
- Automated Test Input Generation for Android: Are We There Yet? (E)2015
- FlowDroidACM SIGPLAN Notices, 2014
- Dynodroid: an input generation system for Android apps2013
- Experiences of System-Level Model-Based GUI Testing of an Android Application2011