Code Question Answering via Task-Adaptive Sequence-to-Sequence Pre-training
Authors:
- 1 December 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 229-238
Abstract
The development of a question answering (QA) system for code can greatly facilitate programs understanding for developers. Recently, pre-trained language models (PLMs) have shown promising results in the code QA task. However, directly applying PLMs to code QA often causes suboptimal performance due to the large discrepancy between pre-training and the downstream QA task. While code PLMs are pre-trained on largescale unlabeled code corpora, there is often a scarce availability of annotated QA pairs for fine-tuning. Existing code PLMs simply reuse the code representation part and require to train the QA part from scratch, which causes the model to overfit QA data. In this paper, we propose CodeMaster, a novel pre-training based approach for automatically answering code questions via task adaptation. CodeMaster employs CodeT5, a popular PLM for source code. In order to mitigate the gap between pretraining and QA, CodeMaster continually pre-trains CodeT5 on multiple self-supervised learning tasks such as partial comment completion and noun-phrase prediction. Experimental results on the CodeQA benchmark show that CodeMaster achieves state-of-the-art performance, and highlight the effectiveness of our approach.Keywords
Funding Information
- National Natural Science Foundation of China
- Tencent
This publication has 21 references indexed in Scilit:
- OpenAPI Bot: A Chatbot to Help You Understand REST APIs2020
- MSRBot: Using bots to answer questions from software repositoriesEmpirical Software Engineering, 2020
- Intermediate-Task Transfer Learning with Pretrained Language Models: When and Why Does It Work?2020
- Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks2020
- CocoQa: Question Answering for Coding Conventions Over Knowledge Graphs2019
- Constructing a Knowledge Base of Coding Conventions from Online Resources2019
- Summarizing Source Code with Transferred API Knowledge2018
- APIBot: Question answering bot for API documentation2017
- AnswerBot: Automated generation of answer summary to developers' technical questions2017
- BLEU2001