AST-trans
Authors:
Open Access
- 21 May 2022
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 150-162
Abstract
Code summarization aims to generate brief natural language descriptions for source codes. The state-of-the-art approaches follow a transformer-based encoder-decoder architecture. As the source code is highly structured and follows strict grammars, its Abstract Syntax Tree (AST) is widely used for encoding structural information. However, ASTs are much longer than the corresponding source code. Existing approaches ignore the size constraint and simply feed the whole linearized AST into the encoders. We argue that such a simple process makes it difficult to extract the truly useful dependency relations from the overlong input sequence. It also incurs significant computational overhead since each node needs to apply self-attention to all other nodes in the AST. To encode the AST more effectively and efficiently, we propose AST-Trans in this paper which exploits two types of node relationships in the AST: ancestor-descendant and sibling relationships. It applies the tree-structured attention to dynamically allocate weights for relevant nodes and exclude irrelevant nodes based on these two relationships. We further propose an efficient implementation to support fast parallel computation for tree-structure attention. On the two code summarization datasets, experimental results show that AST-Trans significantly outperforms the state-of-the-arts while being times more efficient than standard transformers1.Keywords
Funding Information
- NSF award (2034508)
- Natural Science Foundation of Jiangsu Province, China (BK20201250)
- Cooperation Fund of Huawei-NJU Creative Laboratory for the Next Programming
- National Natural Science Foundation of China (61802167, 61802095)
This publication has 30 references indexed in Scilit:
- Estimation of Gap Between Current Language Models and Human Performance2017
- Rethinking the Inception Architecture for Computer Vision2016
- Tree-to-Sequence Attentional Neural Machine Translation2016
- Summarizing Source Code using a Neural Attention Model2016
- Automatic Source Code Summarization of Context for Java MethodsIEEE Transactions on Software Engineering, 2015
- Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks2015
- Automatic generation of natural language summaries for Java classes2013
- On the Use of Automated Text Summarization Techniques for Summarizing Source Code2010
- Towards automatically generating summary comments for Java methods2010
- Recursive distributed representationsArtificial Intelligence, 1990