Neat: Mobile App Layout Similarity Comparison Based on Graph Convolutional Networks
Open Access
- 10 July 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 104-114
Abstract
A wide variety of device models, screen resolutions and operating systems have emerged with recent advances in mobile devices. As a result, the graphical user interface (GUI) layout in mobile apps has become increasingly complex due to this market fragmentation, with rapid iterations being the norm. Testing page layout issues under these circumstances hence becomes a resource-intensive task, requiring significant manpower and effort due to the vast number of device models and screen resolution adaptations. One of the most challenging issues to cover manually is multi-model and cross-version layout verification for the same GUI page. To address this issue, we propose Neat, a non-intrusive end-to-end mobile app layout similarity measurement tool that utilizes computer vision techniques for GUI element detection, layout feature extraction, and similarity metrics. Our empirical evaluation and industrial application have demonstrated that our approach is effective in improving the efficiency of layout assertion testing and ensuring application quality.Keywords
This publication has 33 references indexed in Scilit:
- Exploring Simple Siamese Representation Learning2021
- Momentum Contrast for Unsupervised Visual Representation Learning2020
- Graph neural networks: A review of methods and applicationsAI Open, 2020
- Humanoid: A Deep Learning-Based Approach to Automated Black-box Android App Testing2019
- MobileNetV2: Inverted Residuals and Linear Bottlenecks2018
- Monkey Says, Monkey Does: Security and Privacy on Voice AssistantsIEEE Access, 2017
- Residual Attention Network for Image Classification2017
- SSD: Single Shot MultiBox Detector2016
- NetVLAD: CNN Architecture for Weakly Supervised Place Recognition2016
- DPM, a fast, accurate Monte Carlo code optimized for photon and electron radiotherapy treatment planning dose calculationsPhysics in Medicine & Biology, 2000