The POLAR Framework: Polar Opposites Enable Interpretability of Pre-Trained Word Embeddings
Authors:
Open Access
- 20 April 2020
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1548-1558
Abstract
We introduce ‘POLAR’ — a framework that adds interpretability to pre-trained word embeddings via the adoption of semantic differentials. Semantic differentials are a psychometric construct for measuring the semantics of a word by analysing its position on a scale between two polar opposites (e.g., cold – hot, soft – hard). The core idea of our approach is to transform existing, pre-trained word embeddings via semantic differentials to a new “polar” space with interpretable dimensions defined by such polar opposites. Our framework also allows for selecting the most discriminative dimensions from a set of polar dimensions provided by an oracle, i.e., an external source. We demonstrate the effectiveness of our framework by deploying it to various downstream tasks, in which our interpretable word embeddings achieve a performance that is comparable to the original word embeddings. We also show that the interpretable dimensions selected by our framework align with human judgement. Together, these results demonstrate that interpretability can be added to word embeddings without compromising performance. Our work is relevant for researchers and engineers interested in interpreting pre-trained word embeddings.Keywords
All Related Versions
This publication has 19 references indexed in Scilit:
- SimLex-999: Evaluating Semantic Models With (Genuine) Similarity EstimationComputational Linguistics, 2015
- Sparse Overcomplete Word Vector Representations2015
- Glove: Global Vectors for Word Representation2014
- Large-scale learning of word relatedness with constraints2012
- Punishment, Justice, and Compliance in Mandatory IT SettingsInformation Systems Research, 2011
- Web-based information service adoption: A comparison of the motivational model and the uses and gratifications theoryDecision Support Systems, 2011
- Placing search in contextACM Transactions on Information Systems, 2002
- The relative success of alternative approaches to strategic information systems planning: an empirical analysisThe Journal of Strategic Information Systems, 1999
- Solving Ill-Conditioned and Singular Linear Systems: A Tutorial on RegularizationSIAM Review, 1998
- Contextual correlates of synonymyCommunications of the ACM, 1965