RepairLLaMA: Efficient Representations and Fine-Tuned Adapters for Program Repair
Top Cited Papers
Open Access
- 18 June 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Software Engineering
- Vol. 51 (8) , 2366-2380
Abstract
Automated Program Repair (APR) has evolved significantly with the advent of Large Language Models (LLMs). Fine-tuning LLMs for program repair is a recent avenue of research, with many dimensions which have not been explored. Existing work mostly fine-tune LLMs with naive code representations and does not scale to frontier models. To address this problem, we propose RepairLLaMA, a novel program repair approach that 1) identifies optimal code representations for APR with fine-tuned models, and 2) pioneers state-of-the-art parameter-efficient fine-tuning technique (PEFT) for program repair. This results in RepairLLaMA producing a highly effective ‘program repair adapter’ for fixing bugs with AI. Our experiments demonstrate the validity of both concepts. First, fine-tuning adapters with program repair specific code representations enables the model to use meaningful repair signals and produce better patches. Second, parameter-efficient fine-tuning helps fine-tuning to converge and clearly contributes to the effectiveness of RepairLLaMA in fixing bugs outside the fine-tuning data distribution. Overall, RepairLLaMA correctly fixes 144 Defects4J v2, 109 HumanEval-Java, and 20 GitBug-Java bugs, outperforming all baselines.Keywords
All Related Versions
Funding Information
- Wallenberg Ai, Autonomous Systems and Software Program
- Knut och Alice Wallenbergs Stiftelse
This publication has 45 references indexed in Scilit:
- Program Repair with Minimal Edits Using CodeT52023
- The Plastic Surgery Hypothesis in the Era of Large Language Models2023
- An Empirical Study on Fine-Tuning Large Language Models of Code for Automated Program Repair2023
- Better Patching Using LLM Prompting, via Self-Consistency2023
- A controlled experiment of different code representations for learning-based program repairEmpirical Software Engineering, 2022
- RepairNet: Contextual Sequence-to-Sequence Network for Automated Program Repair2021
- CURE: Code-Aware Neural Machine Translation for Automatic Program Repair2021
- Harnessing Evolution for Multi-Hunk Program Repair2019
- SPOON: A library for implementing analyses and transformations of Java source codeSoftware: Practice and Experience, 2015
- May the source be with youWork Study, 2000