Improving Stack Overflow question title generation with copying enhanced CodeBERT model and bi-modal information
Preprint
Abstract
Context: Stack Overflow is very helpful for software developers who are seeking answers to programming problems. Previous studies have shown that a growing number of questions are of low quality and thus obtain less attention from potential answerers. Gao et al. proposed an LSTM-based model (i.e., BiLSTM-CC) to automatically generate question titles from the code snippets to improve the question quality. However, only using the code snippets in the question body cannot provide sufficient information for title generation, and LSTMs cannot capture the long-range dependencies between tokens. Objective: This paper proposes CCBERT, a deep learning based novel model to enhance the performance of question title generation by making full use of the bi-modal information of the entire question body. Method: CCBERT follows the encoder-decoder paradigm and uses CodeBERT to encode the question body into hidden representations, a stacked Transformer decoder to generate predicted tokens, and an additional copy attention layer to refine the output distribution. Both the encoder and decoder perform the multi-head self-attention operation to better capture the long-range dependencies. This paper builds a dataset containing around 200,000 high-quality questions filtered from the data officially published by Stack Overflow to verify the effectiveness of the CCBERT model. Results: CCBERT outperforms all the baseline models on the dataset. Experiments on both code-only and low-resource datasets show the superiority of CCBERT with less performance degradation. The human evaluation also shows the excellent performance of CCBERT concerning both readability and correlation criteria.Keywords
All Related Versions
This publication has 29 references indexed in Scilit:
- Graph4Web: A relation-aware graph attention network for web service classificationJournal of Systems and Software, 2022
- A comprehensive investigation of the impact of feature selection techniques on crashing fault residence prediction modelsInformation and Software Technology, 2021
- How do developers discuss and support new programming languages in technical Q&A site? An empirical study of Go, Swift, and Rust in Stack OverflowInformation and Software Technology, 2021
- What skills do IT companies look for in new developers? A study with Stack Overflow jobsInformation and Software Technology, 2021
- Generating Question Titles for Stack Overflow from Mined Code SnippetsACM Transactions on Software Engineering and Methodology, 2020
- A large scale study on how developers discuss code smells and anti-pattern in Stack Exchange sitesInformation and Software Technology, 2020
- Mining API usage scenarios from stack overflowInformation and Software Technology, 2020
- How Do Users Revise Answers on Technical Q&A Websites? A Case Study on Stack OverflowIEEE Transactions on Software Engineering, 2018
- How to ask for technical help? Evidence-based guidelines for writing questions on Stack OverflowInformation and Software Technology, 2018
- The Automatic Creation of Literature AbstractsIBM Journal of Research and Development, 1958