Copiloting the Copilots for Automated Program Repair
Open Access
- 30 March 2026
- journal article
- Published by Association for Computing Machinery (ACM) in Communications of the ACM
Abstract
During Automated Program Repair (APR), it can be challenging to synthesize correct patches for real-world systems in general-purpose programming languages. Recent large language models (LLMs) have been shown to be helpful “copilots” in assisting developers with various coding tasks, and have also been directly applied for patch synthesis. However, most LLMs treat programs as sequences of tokens, meaning that they are ignorant of the underlying semantics constraints of the target programming language. This results in plenty of statically invalid generated patches, impeding the practicality of the technique. Therefore, we propose Repilot , a general code generation framework to further copilot the AI “copilots” (i.e., LLMs) by synthesizing more valid patches during the repair process. Our key insight is that many LLMs produce outputs autoregressively (i.e., token by token), resembling human writing programs, which can be significantly boosted and guided through a Completion Engine. Repilot synergistically synthesizes a candidate patch through the interaction between an LLM and a Completion Engine, which 1) prunes away infeasible tokens suggested by the LLM and 2) proactively completes the token based on the suggestions provided by the Completion Engine. Our evaluation on a subset of the widely-used Defects4j 1.2 and 2.0 datasets shows that Repilot outperforms state-of-the-art techniques by fixing 27% and 47% more bugs, respectively. Moreover, Repilot produces more valid and correct patches than the base LLM with the same budget. While we focus on leveraging Repilot for APR in this work, the overall approach is also generalizable to other code generation tasks.Keywords
This publication has 11 references indexed in Scilit:
- Large Language Models Are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Models2023
- Automated Program Repair in the Era of Large Pre-trained Language Models2023
- RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation2023
- Less training, more repairing please: revisiting automated program repair via zero-shot learning2022
- Neural program repair with execution-based backpropagation2022
- A syntax-guided edit decoder for neural program repair2021
- CURE: Code-Aware Neural Machine Translation for Automatic Program Repair2021
- CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation2021
- Automatic Software Repair: A SurveyIEEE Transactions on Software Engineering, 2017
- Defects4J: a database of existing faults to enable controlled testing studies for Java programs2014