Automated Prompt Engineering for Cost-Effective Code Generation Using Evolutionary Algorithms
Open Access
- 18 November 2026
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 35 (12) , 1-26
Abstract
Large Language Models (LLMs) have seen increasing use in various software development tasks, especially in code generation. The most advanced recent methods attempt to incorporate feedback from code execution into prompts to help guide LLMs in generating correct code in an iterative process. While effective, these methods could be costly due to numerous interactions with the LLM and extensive token usage. To address this issue, we propose an alternative approach named Evolutionary Prompt Engineering for Code ( EPiC ) , which leverages a lightweight evolutionary algorithm to refine the original prompts into improved versions that generate high-quality code, with minimal interactions with the LLM. Our evaluation against State-of-the-Art (SOTA) LLM-based code generation agents shows that EPiC not only achieves up to 5% improvement in pass@k but is also 1.2–10 times more cost-effective than the baselines.Keywords
All Related Versions
This publication has 23 references indexed in Scilit:
- Mapping from Meaning: Addressing the Miscalibration of Prompt-Sensitive Language ModelsProceedings of the AAAI Conference on Artificial Intelligence, 2025
- When Punctuation Matters: A Large-Scale Comparison of Prompt Robustness Methods for LLMs2025
- Self-Collaboration Code Generation via ChatGPTACM Transactions on Software Engineering and Methodology, 2024
- Effective test generation using pre-trained Large Language Models and mutation testingInformation and Software Technology, 2024
- An initial investigation of ChatGPT unit test generation capability2023
- Large Language Models for Software Engineering: Survey and Open Problems2023
- Automatic Prompt Optimization with “Gradient Descent” and Beam Search2023
- PRCBERT: Prompt Learning for Requirement Classification using BERT-based Pretrained Language Models2022
- CodeBERT: A Pre-Trained Model for Programming and Natural Languages2020
- A comparative evaluation of genetic and gradient-based algorithms applied to aerodynamic optimizationEuropean Journal of Computational Mechanics, 2008