Shan Gao
Research Subjects
Research Interests
Shan Gao has a total of 6 co-authors. Here are the top co-authors by number of shared publications:
- Chaozheng Wang
The Chinese University of Hong Kong, Hong Kong, China
1 shared publication - Chun Yong Chong
School of Information Technology Monash University Malaysia Malaysia
1 shared publication - Cuiyun Gao
Harbin Institute of Technology, Shenzhen, China
1 shared publication - Michael R. Lyu
The Chinese University of Hong Kong
1 shared publication - Shuzheng Gao
Department of Computer Science and Engineering The Chinese University of Hong Kong China
1 shared publication Xiaoqian Jiao
Harbin Institute of Technology Shenzhen China
1 shared publication
Latest Publications
- Open Access
The Prompt Alchemist: Automated LLM-Tailored Prompt Optimization for Test Case Generation
Test cases are essential for validating the reliability and quality of software applications. Recent studies have demonstrated the capability of Large Language Models (LLMs) to generate useful test cases for given source code. However, the existing work primarily relies on human-written plain prompts, which often leads to suboptimal results since the performance of LLMs can be highly influenced by the prompts. Moreover, these approaches use the same prompt for all LLMs, overlooking the fact that different LLMs might be best suited to different prompts. Given the wide variety of possible prompt formulations, automatically discovering the optimal prompt for each LLM presents a significant challenge. Although there are methods on automated prompt optimization in the natural language processing field, they are hard to produce effective prompts for the test case generation task. First, the methods iteratively optimize prompts by simply combining and mutating existing ones without proper guidance, resulting in prompts that lack diversity and tend to repeat the same errors in the generated test cases. Second, the prompts are generally lack of domain contextual knowledge, limiting LLMs' performance in the task.
Open AccessThe Prompt Alchemist: Automated LLM-Tailored Prompt Optimization for Test Case Generation
Test cases are essential for validating the reliability and quality of software applications. Recent studies have demonstrated the capability of Large Language Models (LLMs) to generate useful test cases for given source code. However, the existing work primarily relies on human-written plain prompts, which often leads to suboptimal results since the performance of LLMs can be highly influenced by the prompts. Moreover, these approaches use the same prompt for all LLMs, overlooking the fact that different LLMs might be best suited to different prompts. Given the wide variety of possible prompt formulations, automatically discovering the optimal prompt for each LLM presents a significant challenge. Although there are methods on automated prompt optimization in the natural language processing field, they are hard to produce effective prompts for the test case generation task. First, the methods iteratively optimize prompts by simply combining and mutating existing ones without proper guidance, resulting in prompts that lack diversity and tend to repeat the same errors in the generated test cases. Second, the prompts are generally lack of domain contextual knowledge, limiting LLMs' performance in the task.