Language Models Implement Simple Word2Vec-style Vector Arithmetic
Top Cited Papers
Open Access
- 1 January 2024
- conference paper
- Published by Association for Computational Linguistics (ACL)
- p. 5030-5047
Abstract
Jack Merullo, Carsten Eickhoff, Ellie Pavlick. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- Dissecting Recall of Factual Associations in Auto-Regressive Language Models2023
- Rethinking the Role of Scale for In-Context Learning: An Interpretability-based Case Study at 66 Billion Scale2023
- When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Data2022
- Transformer Feed-Forward Layers Are Key-Value Memories2021
- Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Bases2021
- Static Embeddings as Efficient Knowledge Bases?2021
- Inducing Relational Knowledge from BERTProceedings of the AAAI Conference on Artificial Intelligence, 2020
- Attention is Not Only a Weight: Analyzing Transformers with Vector Norms2020
- Language Models as Knowledge Bases?2019
- The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives2019