CCISolver: End-to-End Detection and Repair of Method-Level Code-Comment Inconsistency
Preprint
Abstract
Comments within code serve as a crucial foundation for software documentation, facilitating developers to communicate and understand the code effectively. However, code-comment inconsistency (CCI) can negatively affect software development, testing, and maintenance. Recent efforts to mitigate this issue have emerged, but existing studies often suffer from inaccurate datasets and inadequate solutions, weakening their practical effectiveness. In this study, we first conduct a quantitative analysis of existing datasets, revealing a substantial portion of sampled data are mislabeled. To address these data limitations, we introduce CCIBench, a refined dataset comprising high-quality data, to support the training and evaluation of method-level CCI methods. Furthermore, we present an innovative end-to-end LLM-based framework, CCISolver, designed to improve code quality by identifying and rectifying CCIs. Comprehensive evaluations demonstrate CCISolver's superior performance. For detection, it establishes a new state-of-the-art with an F1-score of 89.54%. In fixing task, it achieves a remarkable 18.84% relative improvement in GLEU score over the strongest baseline. This superiority is confirmed by human evaluation, where CCISolver's fixing success rate of 0.6533 significantly surpasses existing methods. Critically, in a practical end-to-end setting, CCISolver's innovative architecture is approximately 36% faster for inference than the baseline model, underscoring its scalability and real-world applicability.Keywords
All Related Versions
This publication has 20 references indexed in Scilit:
- AugGPT: Leveraging ChatGPT for Text Data AugmentationIEEE Transactions on Big Data, 2025
- Exploring the Effectiveness of LLMs in Automated Logging Statement Generation: An Empirical StudyIEEE Transactions on Software Engineering, 2024
- Taxonomy of inline code comment smellsEmpirical Software Engineering, 2024
- Automatic Commit Message Generation: A Critical Review and Directions for Future WorkIEEE Transactions on Software Engineering, 2024
- Code Comment Inconsistency Detection Based on Confidence LearningIEEE Transactions on Software Engineering, 2024
- Predictive Comment Updating With Heuristics and AST-Path-Based Neural Learning: A Two-Phase ApproachIEEE Transactions on Software Engineering, 2022
- Just-In-Time Obsolete Comment Detection and UpdateIEEE Transactions on Software Engineering, 2021
- On the relationship between comment update practices and Software BugsJournal of Systems and Software, 2012
- Learning a Metric for Code ReadabilityIEEE Transactions on Software Engineering, 2009
- The Kappa Statistic in Reliability Studies: Use, Interpretation, and Sample Size RequirementsPhysical Therapy, 2005