The POLAR Framework: Polar Opposites Enable Interpretability of Pre-Trained Word Embeddings
Authors:
- Binny MathewBMBinny Mathew
- Sandipan SikdarSSSandipan Sikdar
- Florian LemmerichFLFlorian Lemmerich
- Markus StrohmaierMSMarkus Strohmaier
Preprint
Abstract
We introduce POLAR - a framework that adds interpretability to pre-trained word embeddings via the adoption of semantic differentials. Semantic differentials are a psychometric construct for measuring the semantics of a word by analysing its position on a scale between two polar opposites (e.g., cold -- hot, soft -- hard). The core idea of our approach is to transform existing, pre-trained word embeddings via semantic differentials to a new "polar" space with interpretable dimensions defined by such polar opposites. Our framework also allows for selecting the most discriminative dimensions from a set of polar dimensions provided by an oracle, i.e., an external source. We demonstrate the effectiveness of our framework by deploying it to various downstream tasks, in which our interpretable word embeddings achieve a performance that is comparable to the original word embeddings. We also show that the interpretable dimensions selected by our framework align with human judgement. Together, these results demonstrate that interpretability can be added to word embeddings without compromising performance. Our work is relevant for researchers and engineers interested in interpreting pre-trained word embeddings.Keywords
All Related Versions
This publication has 20 references indexed in Scilit:
- SPINE: SParse Interpretable Neural EmbeddingsProceedings of the AAAI Conference on Artificial Intelligence, 2018
- Image Captioning with Semantic Attention2016
- SimLex-999: Evaluating Semantic Models With (Genuine) Similarity EstimationComputational Linguistics, 2015
- Bilingual Word Embeddings for Phrase-Based Machine Translation2013
- Large-scale learning of word relatedness with constraints2012
- Punishment, Justice, and Compliance in Mandatory IT SettingsInformation Systems Research, 2011
- Web-based information service adoption: A comparison of the motivational model and the uses and gratifications theoryDecision Support Systems, 2011
- Web performance scaleInformation & Management, 2005
- Solving Ill-Conditioned and Singular Linear Systems: A Tutorial on RegularizationSIAM Review, 1998
- Contextual correlates of synonymyCommunications of the ACM, 1965