A Mechanism for Solving Relational Tasks in Transformer Language Models
Preprint
Abstract
A primary criticism towards language models (LMs) is their inscrutability. This paper presents evidence that, despite their size and complexity, LMs sometimes exploit a simple computational mechanism to solve one-to-one relational tasks (e.g., capital_of(Poland)=Warsaw). We investigate a range of language model sizes (from 124M parameters to 176B parameters) in an in-context learning setting, and find that for a variety of tasks (involving capital cities, upper-casing, and past-tensing) a key part of the mechanism reduces to a simple linear update typically applied by the feedforward (FFN) networks. These updates also tend to promote the output of the relation in a content-independent way (e.g., encoding Poland:Warsaw::China:Beijing), revealing a predictable pattern that these models take in solving these tasks. We further show that this mechanism is specific to tasks that require retrieval from pretraining memory, rather than retrieval from local context. Our results contribute to a growing body of work on the mechanistic interpretability of LLMs, and offer reason to be optimistic that, despite the massive and non-linear nature of the models, the strategies they ultimately use to solve tasks can sometimes reduce to familiar and even intuitive algorithms.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- Dissecting Recall of Factual Associations in Auto-Regressive Language Models2023
- Rethinking the Role of Scale for In-Context Learning: An Interpretability-based Case Study at 66 Billion Scale2023
- Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space2022
- When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Data2022
- Transformer Feed-Forward Layers Are Key-Value Memories2021
- Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Bases2021
- Static Embeddings as Efficient Knowledge Bases?2021
- Inducing Relational Knowledge from BERTProceedings of the AAAI Conference on Artificial Intelligence, 2020
- Attention is Not Only a Weight: Analyzing Transformers with Vector Norms2020
- Language Models as Knowledge Bases?2019