Interpreting and Exploiting Functional Specialization in Multi-Head Attention under Multi-task Learning
Preprint
Abstract
Transformer-based models, even though achieving super-human performance on several downstream tasks, are often regarded as a black box and used as a whole. It is still unclear what mechanisms they have learned, especially their core module: multi-head attention. Inspired by functional specialization in the human brain, which helps to efficiently handle multiple tasks, this work attempts to figure out whether the multi-head attention module will evolve similar function separation under multi-tasking training. If it is, can this mechanism further improve the model performance? To investigate these questions, we introduce an interpreting method to quantify the degree of functional specialization in multi-head attention. We further propose a simple multi-task training method to increase functional specialization and mitigate negative information transfer in multi-task learning. Experimental results on seven pre-trained transformer models have demonstrated that multi-head attention does evolve functional specialization phenomenon after multi-task training which is affected by the similarity of tasks. Moreover, the multi-task training strategy based on functional specialization boosts performance in both multi-task learning and transfer learning without adding any parameters.Keywords
All Related Versions
This publication has 31 references indexed in Scilit:
- Differentiable Subset Pruning of Transformer HeadsTransactions of the Association for Computational Linguistics, 2021
- Why Attentions May Not Be Interpretable?2021
- Unsupervised neural network models of the ventral visual streamProceedings of the National Academy of Sciences of the United States of America, 2021
- Learning Sparse Sharing Architectures for Multiple TasksProceedings of the AAAI Conference on Artificial Intelligence, 2020
- Roles and Utilization of Attention Heads in Transformer-based Neural Language Models2020
- Towards Faithfully Interpretable NLP Systems: How Should We Define and Evaluate Faithfulness?2020
- Toward a universal decoder of linguistic meaning from brain activationNature Communications, 2018
- SciTaiL: A Textual Entailment Dataset from Science Question AnsweringProceedings of the AAAI Conference on Artificial Intelligence, 2018
- Functional specificity in the human brain: A window into the functional architecture of the mindProceedings of the National Academy of Sciences, 2010
- From Neuropsychology to Mental Structure1988