Predictive Comment Updating With Heuristics and AST-Path-Based Neural Learning: A Two-Phase Approach
Open Access
- 27 June 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Software Engineering
- Vol. 49 (4) , 1640-1660
Abstract
Just-in-time comment update is a promising way to reduce the burden of developers during software maintenance and evolution. Existing approaches can be divided into two categories: the heuristic-based approach and the deep-learning-based approach. The heuristic-based approach is restricted to a specific type of comment updates (i.e., code-indicative updates), but performs well on such type. The effectiveness of deep-learning-based approach is limited but it can handle diverse comment updates. Considering the complementary advantages of existing approaches, an intuitive idea is to combine them for better performance. To investigate this idea, we first conduct a pre-study experiment which shows that to construct an effective comment updater by combining heuristic-based and deep-learning-based approaches, we need to tackle two main challenges: 1) the heuristic-based approach may bring side effects to cases which cannot be updated by it; and 2) the current deep-learning-based approach is with limited effectiveness. Then, we propose a novel two-phase approach named Toper to cope with these two challenges and effectively perform comment updates. In the first phase, Toper integrates nine distinctive features identified through our large-scale empirical analysis into a predictive model, which can predict whether the contents of the comment updates can be found in the corresponding code changes, namely, the comment updates are code-indicative updates. If so, the updates are then generated by an off-the-shelf heuristic-based approach; otherwise, Toper leverages a deep learning model, which we specially designed for non-code-indicative updates, to infer the new comment based on the old comment and code change. Motivated by our manual observation on the limitation of existing approaches on non-code-indicative updates, our deep learning model adopts the Abstract Syntax Tree path technique, which can capture the program structure information for effectively embedding code changes. Our evaluation shows that our approach outperforms the state-of-the-art by around 20% with respect to the number of correct comments it generates. Via in-depth analysis, we illustrate the rationale of each design decision as well as point out potential directions.Keywords
Funding Information
- National Natural Science Foundation of China (61872445, 61672529)
- Zhejiang University Education Foundation Qizhen Scholar Foundation
This publication has 65 references indexed in Scilit:
- Revisiting the Impact of Classification Techniques on the Performance of Defect Prediction Models2015
- From source code identifiers to natural language termsJournal of Systems and Software, 2014
- Fine-grained and accurate source code differencing2014
- On the relationship between comment update practices and Software BugsJournal of Systems and Software, 2012
- Towards automatically generating summary comments for Java methods2010
- TODO or to bug2008
- /*icomment2007
- Does the modern code inspection have value?2002
- On a Test of Whether one of Two Random Variables is Stochastically Larger than the OtherThe Annals of Mathematical Statistics, 1947
- Individual Comparisons by Ranking MethodsBiometrics Bulletin, 1945