Improving Web Element Localization by Using a Large Language Model
Authors:
Open Access
- 15 August 2024
- journal article
- research article
- Published by Wiley in Software Testing, Verification and Reliability
- Vol. 34 (7)
Abstract
Web‐based test automation heavily relies on accurately finding web elements. Traditional methods compare attributes but do not grasp the context and meaning of elements and words. The emergence of large language models (LLMs) like GPT‐4, which can show human‐like reasoning abilities on some tasks, offers new opportunities for software engineering and web element localization. This paper introduces and evaluates VON Similo LLM, an enhanced web element localization approach. Using an LLM, it selects the most likely web element from the top‐ranked ones identified by the existing VON Similo method, ideally aiming to get closer to human‐like selection accuracy. An experimental study was conducted using 804 web element pairs from 48 real‐world web applications. We measured the number of correctly identified elements as well as the execution times, comparing the effectiveness and efficiency of VON Similo LLM against the baseline algorithm. In addition, motivations from the LLM were recorded and analysed for 140 instances. VON Similo LLM demonstrated improved performance, reducing failed localizations from 70 to 40 (out of 804), a 43% reduction. Despite its slower execution time and additional costs of using the GPT‐4 model, the LLM's human‐like reasoning showed promise in enhancing web element localization. LLM technology can enhance web element localization in GUI test automation, reducing false positives and potentially lowering maintenance costs. However, further research is necessary to fully understand LLMs' capabilities, limitations and practical use in GUI testing.Keywords
All Related Versions
Funding Information
- Stiftelsen för Kunskaps- och Kompetensutveckling
This publication has 36 references indexed in Scilit:
- Why many challenges with GUI test automation (will) remainInformation and Software Technology, 2021
- Sidereal: Statistical adaptive generation of robust locators for web testingSoftware Testing, Verification and Reliability, 2021
- Generating and selecting resilient and maintainable locators for Web automated testingSoftware Testing, Verification and Reliability, 2021
- Robotic process automationElectronic Markets, 2019
- On the long-term use of visual gui testing in industrial practice: a case studyEmpirical Software Engineering, 2017
- Maintenance of automated test suites in industry: An empirical study on Visual GUI TestingInformation and Software Technology, 2016
- Robula+: an algorithm for generating robust XPath locators for web testingJournal of Software: Evolution and Process, 2016
- Design and industrial evaluation of a tool supporting semi-automated website testingSoftware Testing, Verification and Reliability, 2012
- Automated GUI performance testingSoftware Quality Journal, 2011
- Automated browsing in AJAX websitesData & Knowledge Engineering, 2011