Code Comment Inconsistency Detection Based on Confidence Learning
- 29 January 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Software Engineering
- Vol. 50 (3) , 598-617
Abstract
Code comments are a crucial source of software documentation that captures various aspects of the code. Such comments play a vital role in understanding the source code and facilitating communication between developers. However, with the iterative release of software, software projects become larger and more complex, leading to a corresponding increase in issues such as mismatched, incomplete, or outdated code comments. These inconsistencies in code comments can misguide developers and result in potential bugs, and there has been a steady rise in reports of such inconsistencies over time. Despite numerous methods being proposed for detecting code comment inconsistencies, their learning effect remains limited due to a lack of consideration for issues such as characterization noise and labeling errors in datasets. To overcome these limitations, we propose a novel approach called MCCL that first removes noise from the dataset and then detects inconsistent code comments in a timely manner, thereby enhancing the model’s learning ability. Our proposed model facilitates better matching between code and comments, leading to improved development of software engineering projects. MCCL comprises two components, namely method comment detection and confidence learning denoising. The method comment detection component captures the intricate relationships between code and comments by learning their syntactic and semantic structures. It correlates the code and comments through an attention mechanism to identify how changes in the code affect the comments. Furthermore, confidence learning denoising component of MCCL identifies and removes characterization noises and labeling errors to enhance the quality of the datasets. This is achieved by implementing principles such as pruning noisy data, counting with probabilistic thresholds to estimate noise, and ranking examples to train with confidence. By effectively eliminating noise from the dataset, our model is able to more accurately learn inconsistencies between comments and source code. Our experiments on 1,518 open-source projects demonstrate that MCCL can accurately detect inconsistencies, achieving an average F1-score of 82.6%. This result outperforms state-of-the-art methods by 2.4% to 28.0%. Therefore, MCCL is more effective in identifying inconsistent comments based on code changes compared to existing approaches.Keywords
Funding Information
- National Natural Science Foundation of China (62202079, 62032004, 61902050)
- Dalian Excellent Young Project (2022RY35)
- Postgraduate Education Reform Project of Liaoning Province (385-151)
- Fundamental Research Funds for the Central Universities (DUT22ZD101)
- Dalian Maritime University Teacher Development Project (JF2023Y05)
- China Higher Education Association 2023 Higher Education Scientific Research (23LK0408)
This publication has 38 references indexed in Scilit:
- Symmetric Cross Entropy for Robust Learning With Noisy Labels2019
- Classifying code comments in Java software systemsEmpirical Software Engineering, 2019
- Automatic Detection of Outdated Comments During Code Changes2018
- Detecting fragile comments2017
- Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach2017
- Coherence of comments and method implementations: a dataset and an empirical investigationSoftware Quality Journal, 2016
- On the relationship between comment update practices and Software BugsJournal of Systems and Software, 2012
- A Memory Model for Static Analysis of C Programs2010
- Automatic Quality Assessment of Source Code Comments: The JavadocMiner2010
- Analyzing the co-evolution of comments and source codeSoftware Quality Journal, 2009