Unified Instance and Knowledge Alignment Pretraining for Aspect-Based Sentiment Analysis
- 14 July 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 31 (23299290) , 2629-2642
Abstract
The goal of aspect-based sentiment analysis (ABSA) is to determine the sentiment polarity towards an aspect. Because of the expensive and limited amounts of labelled data, the pretraining strategy has become the de facto standard for ABSA. However, there always exists a severe domain shift between the pretraining and downstream ABSA datasets, which hinders effective knowledge transfer when directly fine-tuning, making the downstream task suboptimal. To mitigate this domain shift, we introduce a unified alignment pretraining framework into the vanilla pretrain-finetune pipeline, that has both instance- and knowledge-level alignments. Specifically, we first devise a novel coarse-to-fine retrieval sampling approach to select target domain-related instances from the large-scale pretraining dataset, thus aligning the instances between pretraining and the target domains ( First Stage ). Then, we introduce a knowledge guidance-based strategy to further bridge the domain gap at the knowledge level. In practice, we formulate the model pretrained on the sampled instances into a knowledge guidance model and a learner model. On the target dataset, we design an on-the-fly teacher-student joint fine-tuning approach to progressively transfer the knowledge from the knowledge guidance model to the learner model ( Second Stage ). Therefore, the learner model can maintain more domain-invariant knowledge when learning new knowledge from the target dataset. In the Third Stage, the learner model is finetuned to better adapt its learned knowledge to the target dataset. Extensive experiments and analyses on several ABSA benchmarks demonstrate the effectiveness and universality of our proposed pretraining framework.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62225113, 62076186)
- National Key Research and Development Program of China (2021YFB2401302)
This publication has 57 references indexed in Scilit:
- Sentence Compression for Aspect-Based Sentiment AnalysisIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2015
- SemEval-2015 Task 12: Aspect Based Sentiment Analysis2015
- Document Modeling with Gated Recurrent Neural Network for Sentiment Classification2015
- Adaptive Recursive Neural Network for Target-dependent Twitter Sentiment Classification2014
- Learning Sentiment-Specific Word Embedding for Twitter Sentiment Classification2014
- SemEval-2014 Task 4: Aspect Based Sentiment Analysis2014
- Glove: Global Vectors for Word Representation2014
- A Survey of Opinion Mining and Sentiment Analysis2012
- The Probabilistic Relevance Framework: BM25 and BeyondFoundations and Trends® in Information Retrieval, 2009
- Mining and summarizing customer reviews2004