TestWeaver: Execution-aware, Feedback-driven Regression Testing Generation with Large Language Models
Preprint
Abstract
Regression testing ensures that code changes do not unintentionally break existing functionality. While recent advances in large language models (LLMs) have shown promise in automating test generation for regression testing, they often suffer from limited reasoning about program execution, resulting in stagnated coverage growth - a phenomenon known as the coverage plateau. In this paper, we present TestWeaver, a novel LLM-based approach that integrates lightweight program analysis to guide test generation more effectively. TestWeaver introduces three key innovations: (1) it reduces hallucinations and improves focus by supplying the LLM with the backward slice from the target line instead of full program context; (2) it identifies and incorporates close test cases - those that share control-flow similarities with the path to the target line - to provide execution context within the LLM's context window; and (3) it enhances LLM's reasoning with execution in-line annotations that encode variable states as comments along executed paths. By equipping LLMs with these targeted and contextualized inputs, TestWeaver improves coverage-guided test generation and mitigates redundant explorations. Empirical results demonstrate that TestWeaver accelerates code coverage growth and generates more effective regression test cases than existing LLM-based approaches.Keywords
All Related Versions
This publication has 19 references indexed in Scilit:
- Reasoning Runtime Behavior of a Program with LLM: How Far are We?2025
- Planning a Large Language Model for Static Detection of Runtime Errors in Code Snippets2025
- LLM-Based Test-Driven Interactive Code Generation: User Study and Empirical EvaluationIEEE Transactions on Software Engineering, 2024
- Code-Aware Prompting: A Study of Coverage-Guided Test Generation in Regression Setting using LLMProceedings of the ACM on Software Engineering, 2024
- Effective test generation using pre-trained Large Language Models and mutation testingInformation and Software Technology, 2024
- Software Testing With Large Language Models: Survey, Landscape, and VisionIEEE Transactions on Software Engineering, 2024
- An Empirical Evaluation of Using Large Language Models for Automated Unit Test GenerationIEEE Transactions on Software Engineering, 2023
- CodaMosa: Escaping Coverage Plateaus in Test Generation with Pre-trained Large Language Models2023
- CUTE2005
- DART2005