Refining Word Embeddings Using Intensity Scores for Sentiment Analysis
- 29 December 2017
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 26 (3) , 671-681
Abstract
Word embeddings that provide continuous low-dimensional vector representations of words have been extensively used for various natural language processing tasks. However, existing context-based word embeddings such as Word2vec and GloVe typically fail to capture sufficient sentiment information, which may result in words with similar vector representations having an opposite sentiment polarity (e.g., good and bad), thus degrading sentiment analysis performance. To tackle this problem, recent studies have suggested learning sentiment embeddings to incorporate the sentiment polarity (positive and negative) information from labeled corpora. This study adopts another strategy to learn sentiment embeddings. Instead of creating a new word embedding from labeled corpora, we propose a word vector refinement model to refine existing pretrained word vectors using real-valued sentiment intensity scores provided by sentiment lexicons. The idea of the refinement model is to improve each word vector such that it can be closer in the lexicon to both semantically and sentimentally similar words (i.e., those with similar intensity scores) and further away from sentimentally dissimilar words (i.e., those with dissimilar intensity scores). An obvious advantage of the proposed method is that it can be applied to any pretrained word embeddings. In addition, the intensity scores can provide more fine-grained (real-valued) sentiment information than binary polarity labels to guide the refinement process. Experimental results show that the proposed refinement model can improve both conventional word embeddings and previously proposed sentiment embeddings for binary, ternary, and fine-grained sentiment classification on the SemEval and Stanford Sentiment Treebank datasets.Keywords
Funding Information
- Ministry of Science and Technology, Taiwan (MOST 105-2221-E-155-059-MY2 and MOST 105-2218-E-006-028)
This publication has 46 references indexed in Scilit:
- Valence and Arousal Ratings for 420 Finnish Nouns by Age and GenderPLOS ONE, 2013
- Norms of valence, arousal, dominance, and age of acquisition for 4,300 Dutch wordsBehavior Research Methods, 2012
- CROWDSOURCING A WORD–EMOTION ASSOCIATION LEXICONComputational Intelligence, 2012
- Valence, arousal, and imagery ratings for 835 French attributes by young, middle-aged, and older adults: The French Emotional Evaluation List (FEEL)European Review of Applied Psychology, 2012
- Sentiment strength detection for the social webJournal of the American Society for Information Science and Technology, 2011
- Lexicon-Based Methods for Sentiment AnalysisComputational Linguistics, 2011
- Composition in Distributional Models of SemanticsCognitive Science, 2010
- The Berlin Affective Word List Reloaded (BAWL-R)Behavior Research Methods, 2009
- Explorations in context space: Words, sentences, discourseDiscourse Processes, 1998
- WordNetCommunications of the ACM, 1995