SKEP: Sentiment Knowledge Enhanced Pre-training for Sentiment Analysis
Preprint
Abstract
Recently, sentiment analysis has seen remarkable advance with the help of pre-training approaches. However, sentiment knowledge, such as sentiment words and aspect-sentiment pairs, is ignored in the process of pre-training, despite the fact that they are widely used in traditional sentiment analysis approaches. In this paper, we introduce Sentiment Knowledge Enhanced Pre-training (SKEP) in order to learn a unified sentiment representation for multiple sentiment analysis tasks. With the help of automatically-mined knowledge, SKEP conducts sentiment masking and constructs three sentiment knowledge prediction objectives, so as to embed sentiment information at the word, polarity and aspect level into pre-trained sentiment representation. In particular, the prediction of aspect-sentiment pairs is converted into multi-label classification, aiming to capture the dependency between words in a pair. Experiments on three kinds of sentiment tasks show that SKEP significantly outperforms strong pre-training baseline, and achieves new state-of-the-art results on most of the test datasets. We release our code at https://github.com/baidu/Senta.Keywords
All Related Versions
This publication has 20 references indexed in Scilit:
- Modeling sentiment dependencies with graph convolutional networks for aspect-level sentiment classificationKnowledge-Based Systems, 2020
- SenseBERT: Driving Some Sense into BERT2020
- A Knowledge Regularized Hierarchical Approach for Emotion Cause Analysis2019
- GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge2019
- Deep learning for sentiment analysis: A surveyWIREs Data Mining and Knowledge Discovery, 2018
- Recurrent Neural Networks with Auxiliary Labels for Cross-Domain Opinion Target ExtractionProceedings of the AAAI Conference on Artificial Intelligence, 2017
- A Question Answering Approach for Emotion Cause Extraction2017
- Deep Multi-Task Learning for Aspect Term Extraction with Memory Interaction2017
- Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank2013
- Opinion Mining and Sentiment Analysis2008