What Were You Thinking? An LLM-Driven Large-Scale Study of Refactoring Motivations in Open-Source Projects
Authors:
- Mikel RobredoMRMikel Robredo
- Matteo EspositoMEMatteo Esposito
- Fabio PalombaFPFabio Palomba
- Rafael PeñalozaRPRafael Peñaloza
- Valentina LenarduzziVLValentina Lenarduzzi
Preprint
Abstract
Context. Code refactoring improves software quality without changing external behavior. Despite its advantages, its benefits are hindered by the considerable cost of time, resources, and continuous effort it demands. Aim. Understanding why developers refactor, and which metrics capture these motivations, may support wider and more effective use of refactoring in practice. Method. We performed a large-scale empirical study to analyze developers refactoring activity, leveraging Large Language Models (LLMs) to identify underlying motivations from version control data, comparing our findings with previous motivations reported in the literature. Results. LLMs matched human judgment in 80% of cases, but aligned with literature-based motivations in only 47%. They enriched 22% of motivations with more detailed rationale, often highlighting readability, clarity, and structural improvements. Most motivations were pragmatic, focused on simplification and maintainability. While metrics related to developer experience and code readability ranked highest, their correlation with motivation categories was weak. Conclusions. We conclude that LLMs effectively capture surface-level motivations but struggle with architectural reasoning. Their value lies in providing localized explanations, which, when combined with software metrics, can form hybrid approaches. Such integration offers a promising path toward prioritizing refactoring more systematically and balancing short-term improvements with long-term architectural goals.Keywords
All Related Versions
This publication has 36 references indexed in Scilit:
- Exploring the potential of general purpose LLMs in automated software refactoring: an empirical studyAutomated Software Engineering, 2025
- Using AI-based coding assistants in practice: State of affairs, perceptions, and ways forwardInformation and Software Technology, 2024
- Enhancing the defectiveness prediction of methods and classes via JITEmpirical Software Engineering, 2023
- How do i refactor this? An empirical study on refactoring trends and topics in Stack OverflowEmpirical Software Engineering, 2021
- A Brief Review on Multi-objective Software Refactoring and a New Method for Its RecommendationArchives of Computational Methods in Engineering, 2021
- A comprehensive model for code readabilityJournal of Software: Evolution and Process, 2018
- Reviewer recommendation for pull-requests in GitHub: What can we learn from code review and bug assignment?Information and Software Technology, 2016
- Refactoring: Improving the Design of Existing Code2002
- Sample size in qualitative researchResearch in Nursing & Health, 1995
- Note on the Sampling Error of the Difference Between Correlated Proportions or PercentagesPsychometrika, 1947