AutoQALLMs: Automating Web Application Testing Using Large Language Models (LLMs) and Selenium
Open Access
Abstract
Modern web applications change frequently in response to user and market needs, making their testing challenging. Manual testing and automation methods often struggle to keep up with these changes. We propose an automated testing framework, AutoQALLMs, that utilises various LLMs (Large Language Models), including GPT-4, Claude, and Grok, alongside Selenium WebDriver, BeautifulSoup, and regular expressions. This framework enables one-click testing, where users provide a URL as input and receive test results as output, thus eliminating the need for human intervention. It extracts HTML (Hypertext Markup Language) elements from the webpage and utilises the LLMs API to generate Selenium-based test scripts. Regular expressions enhance the clarity and maintainability of these scripts. The scripts are executed automatically, and the results, such as pass/fail status and error details, are displayed to the tester. This streamlined input–output process forms the core foundation of the AutoQALLMs framework. We evaluated the framework on 30 websites. The results show that the system drastically reduces the time needed to create test cases, achieves broad test coverage (96%) with Claude 4.5 LLM, which is competitive with manual scripts (98%), and allows for rapid regeneration of tests in response to changes in webpage structure. Software testing expert feedback confirmed that the proposed AutoQALLMs method for automated web application testing enables faster regression testing, reduces manual effort, and maintains reliable test execution. However, some limitations remain in handling complex page changes and validation. Although Claude 4.5 achieved slightly higher test coverage in the comparative evaluation of the proposed experiment, GPT-4 was selected as the default model for AutoQALLMs due to its cost-efficiency, reproducibility, and stable script generation across diverse websites. Future improvements may focus on increasing accuracy, adding self-healing techniques, and expanding to more complex testing scenarios.Keywords
This publication has 32 references indexed in Scilit:
- Large Language Models for Automated Web-Form-Test Generation: An Empirical StudyACM Transactions on Software Engineering and Methodology, 2026
- Large Language Model for Vulnerability Detection and Repair: Literature Review and the Road AheadACM Transactions on Software Engineering and Methodology, 2025
- Improving Web Element Localization by Using a Large Language ModelSoftware Testing, Verification and Reliability, 2024
- Software Testing With Large Language Models: Survey, Landscape, and VisionIEEE Transactions on Software Engineering, 2024
- An Empirical Evaluation of Using Large Language Models for Automated Unit Test GenerationIEEE Transactions on Software Engineering, 2023
- Web Scraping using Natural Language Processing: Exploiting Unstructured Text for Data Extraction and AnalysisProcedia Computer Science, 2023
- Automating the Detection of Access Control Vulnerabilities in Web ApplicationsSN Computer Science, 2022
- A deep learning-based automated framework for functional User Interface testingInformation and Software Technology, 2022
- Machine Learning Applied to Software Testing: A Systematic Mapping StudyIEEE Transactions on Reliability, 2019
- Web application testing: A systematic literature reviewJournal of Systems and Software, 2014