Knowledge-Enhanced Prompt-Tuning With Contrastive Learning for Emotion Recognition in Conversations
- 24 July 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech and Language Processing
- Vol. 33, 3292-3306
Abstract
Emotion Recognition in Conversation (ERC) is essential for a wide range of applications, but it faces challenges arising from contextual influences. Imbalanced emotion distributions further complicate the ERC task, resulting in constrained learning for minority emotions and biased predictions for common emotions. To address these issues, we propose the Knowledge-Enhanced Prompt-Tuning with Contrastive Learning (KEPTCL) framework. KEPTCL leverages prompt-tuning to align the ERC task with the pre-training objective of the Pre-trained Language Model (PLM), thereby effectively harnessing the rich linguistic knowledge accumulated by the PLM from large-scale corpora. This approach effectively mitigates the data scarcity issue for minority emotions. Furthermore, we introduce a context-aware mixed prompt template and a knowledge subgraph-based label mapping strategy to aid the prompt-tuning process. Moreover, Supervised Contrastive Learning (SCL) is integrated to improve the PLM's capability to distinguish utterances with different emotions, reducing biased predictions for common emotions. Experimental results demonstrate that KEPTCL outperforms the state-of-the-art methods on all three benchmark datasets and mitigates the challenges posed by the imbalanced emotion distributions.Keywords
Funding Information
- National Natural Science Foundation of China (62472092, 62172089, 62106045)
- Natural Science Foundation of Jiangsu Province (BK20241751)
- Jiangsu Provincial Key Laboratory of Computer Networking Technology (BM2003201)
- Key Laboratory of Computer Network and Information Integration of Ministry of Education of China (93K-9)
- Nanjing Purple Mountain Laboratories
This publication has 50 references indexed in Scilit:
- FrameERC: Framelet Transform Based Multimodal Graph Neural Networks for Emotion Recognition in ConversationPattern Recognition, 2025
- Multimodal graph learning with framelet-based stochastic configuration networks for emotion recognition in conversationInformation Sciences, 2024
- CoCoNet: Coupled Contrastive Learning Network with Multi-level Feature Ensemble for Multi-modality Image FusionInternational Journal of Computer Vision, 2023
- Contextual Information and Commonsense Based Prompt for Emotion Recognition in Conversation2023
- Emotion recognition in conversations with emotion shift detection based on multi-task learningKnowledge-Based Systems, 2022
- PTR: Prompt Tuning with Rules for Text ClassificationAI Open, 2022
- A Sequential Contrastive Learning Framework for Robust Dysarthric Speech Recognition2021
- Momentum Contrast for Unsupervised Visual Representation Learning2020
- Neighbourhood sampling in bagging for imbalanced dataNeurocomputing, 2015
- IEMOCAP: interactive emotional dyadic motion capture databaseLanguage Resources and Evaluation, 2008