Optimizing Deep Learning Models to Address Class Imbalance in Code Comment Classification
Authors:
- 27 April 2025
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 45-48
Abstract
Developers rely on code comments to document their work, track issues, and understand the source code. As such, comments provide valuable insights into developers' understanding of their code and describe their various intentions in writing the surrounding code. Recent research leverages natural language processing and deep learning to classify comments based on developers' intentions. However, such labelled data are often imbalanced, causing learning models to perform poorly. This work investigates the use of different weighting strategies of the loss function to mitigate the scarcity of certain classes in the dataset. In particular, various RoBERTa-based transformer models are fine-tuned by means of a hyperparameter search to identify their optimal parameter configurations. Additionally, we fine-tuned the transformers with different weighting strategies for the loss function to address class imbalances. Our approach outperforms the STACC baseline by 8.9 per cent on the NLBSE'25 Tool Competition dataset in terms of the average F1c score, and exceeding the baseline approach in 17 out of 19 cases with a gain ranging from −5.0 to 38.2. The source code is publicly available at https://github.com/moritzmock/NLBSE2025.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- Where Do Developers Admit their Security-Related Concerns?2025
- MADE-WIC: Multiple Annotated Datasets for Exploring Weaknesses In Code2024
- Dopamin: Transformer-based Comment Classifiers through Domain Post-Training and Multi-level Layer Aggregation2024
- STACC: Code Comment Classification using SentenceTransformers2023
- LineVul2022
- WeakSATD2022
- How to identify class comment types? A multi-language approach for class comment classificationJournal of Systems and Software, 2021
- Self-admitted technical debt practices: a comparison between industry and open-sourceEmpirical Software Engineering, 2021
- Classifying Code Comments in Java Open-Source Software Systems2017
- A large-scale empirical study on self-admitted technical debt2016