Source Code Summarization with Structural Relative Position Guided Transformer
Authors:
- 1 March 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 13-24
Abstract
Source code summarization aims at generating concise and clear natural language descriptions for programming languages. Well-written code summaries are beneficial for programmers to participate in the software development and maintenance process. To learn the semantic representations of source code, recent efforts focus on incorporating the syntax structure of code into neural networks such as Transformer. Such Transformer-based approaches can better capture the long-range dependencies than other neural networks including Recurrent Neural Networks (RNNs), however, most of them do not consider the structural relative correlations between tokens, e.g., relative positions in Abstract Syntax Trees (ASTs), which is beneficial for code semantics learning. To model the structural dependency, we propose a StruCtural RelatIve Position guided Transformer, named SCRIPT. SCRIPT first obtains the structural relative positions between tokens via parsing the ASTs of source code, and then passes them into two types of Transformer encoders. One Transformer directly adjusts the input according to the structural relative distance; and the other Transformer encodes the structural relative positions during computing the self-attention scores. Finally, we stack these two types of Transformer encoders to learn representations of source code. Experimental results show that the proposed SCRIPT outperforms the state-of-the-art methods by at least 1.6%, 1.4% and 2.8% with respect to BLEU, ROUGE-L and METEOR on benchmark datasets, respectively. We further show that how the proposed SCRIPT captures the structural relative dependencies.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62002084)
- Shenzhen Science and Technology Innovation Commission (JCYJ20200109113403826)
This publication has 38 references indexed in Scilit:
- A Neural Architecture for Generating Natural Language Descriptions from Source Code Changes2017
- Tree-to-Sequence Attentional Neural Machine Translation2016
- Summarizing Source Code using a Neural Attention Model2016
- CloCom: Mining existing source code for automatic comment generation2015
- Improving automated source code summarization via an eye-tracking study of programmers2014
- Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation2014
- Evaluating source code summarization techniques: Replication and expansion2013
- Using IR methods for labeling source code artifacts: Is it worthwhile?2012
- BLEU2001
- Algorithm 97: Shortest pathCommunications of the ACM, 1962