RAPGen: An Approach for Fixing Code Inefficiencies in Zero-Shot

Performance bugs are non-functional bugs that can even manifest in well-tested commercial products. Fixing these performance bugs is an important yet challenging problem. In this work, we address this challenge and present a new approach called Retrieval-Augmented Prompt Generation (RAPGen). Given a code snippet with a performance issue, RAPGen first retrieves a prompt instruction from a pre-constructed knowledge-base of previous performance bug fixes and then generates a prompt using the retrieved instruction. It then uses this prompt on a Large Language Model in zero-shot to generate a fix. We compare our approach with the various prompt variations and state of the art methods in the task of performance bug fixing. Our empirical evaluation shows that RAPGen can generate performance improvement suggestions equivalent or better than a developer in $\sim 60 {\%}$ of the cases, getting $\sim 42 {\%}$ of them verbatim, in an expert-verified dataset of past performance changes made by C\# developers. Furthermore, we conduct an in-the-wild evaluation to verify the model's effectiveness in practice by suggesting fixes to developers in a large software company. So far, we have shared performance fixes on 10 codebases that represent production services running in the cloud and 7 of the fixes have been accepted by the developers and integrated into the code.