ChatUniTest: A Framework for LLM-Based Test Generation
Top Cited Papers
Open Access
- 10 July 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 572-576
Abstract
Unit testing is an essential yet frequently arduous task. Various automated unit test generation tools have been introduced to mitigate this challenge. Notably, methods based on large language models (LLMs) have garnered considerable attention and exhibited promising results in recent years. Nevertheless, LLM-based tools encounter limitations in generating accurate unit tests. This paper presents ChatUniTest, an LLM-based automated unit test generation framework. ChatUniTest incorporates an adaptive focal context mechanism to encompass valuable context in prompts and adheres to a generation-validation-repair mechanism to rectify errors in generated unit tests. Subsequently, we have developed ChatUniTest Core, a common library that implements core workflow, complemented by the ChatUniTest Toolchain, a suite of seamlessly integrated tools enhancing the capabilities of ChatUniTest. Our effectiveness evaluation reveals that ChatUniTest outperforms TestSpark and EvoSuite in half of the evaluated projects, achieving the highest overall line coverage. Furthermore, insights from our user study affirm that ChatUniTest delivers substantial value to various stakeholders in the software testing domain. ChatUniTest is available at https://github.com/ZJU-ACES-ISE/ChatUniTest, and the demo video is available at https://www.youtube.com/watch?v=GmfxQUqm2ZQ.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (U20A20173,62125206)
- National Key Research and Development Program of China (2022YFF0902702)
- Zhejiang Pioneer (Jianbing) Project (2023C01045)
- Key Research and Development Program of Ningbo (2023Z235)
- ZJU-Hundsun Fintech Research and Development Center ()
This publication has 7 references indexed in Scilit:
- Code-Aware Prompting: A Study of Coverage-Guided Test Generation in Regression Setting using LLMProceedings of the ACM on Software Engineering, 2024
- Software Testing With Large Language Models: Survey, Landscape, and VisionIEEE Transactions on Software Engineering, 2024
- A3test: Assertion-Augmented Automated Test Case GenerationSSRN Electronic Journal, 2024
- Generative Type Inference for Python2023
- CAT-LM Training Language Models on Aligned Code And Tests2023
- Graph-based seed object synthesis for search-based unit testing2021
- EvoSuite2011