Solving Data Imbalance in Text Classification With Constructing Contrastive Samples
Open Access
- 21 August 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 90554-90562
Abstract
Contrastive learning (CL) has been successfully applied in Natural Language Processing (NLP) as a powerful representation learning method and has shown promising results in various downstream tasks. Recent research has highlighted the importance of constructing effective contrastive samples through data augmentation. However, current data augmentation methods primarily rely on random word deletion, substitution, and cropping, which may introduce noisy samples and hinder representation learning. In this article, we propose a novel approach to address data imbalance in text classification by constructing contrastive samples. Our method involves the use of a Label-indicative Component to generate high-quality positive samples for the minority class, along with the introduction of a Hard Negative Mixing strategy to synthesize challenging negative samples at the feature level. By applying supervised contrastive learning to these samples, we are able to obtain superior text representations, which significantly benefit text classification tasks with imbalanced data. Our approach effectively mitigates distributional biases and promotes noise-resistant representation learning. To validate the effectiveness of our method, we conducted experiments on benchmark datasets (THUCNews, AG’s News, 20NG) as well as the imbalanced FDCNews dataset. The code for our method is publicly available at the following GitHub repository: https://github.com/hanggun/CLDMTC.Keywords
Funding Information
- National Key Research and Development Program of China (2022YFF0903302)
This publication has 23 references indexed in Scilit:
- A multi-label text classification method via dynamic semantic representation model and deep neural networkApplied Intelligence, 2020
- MFC-GAN: Class-imbalanced dataset classification using Multiple Fake Class Generative Adversarial NetworkNeurocomputing, 2019
- Bag of Tricks for Efficient Text Classification2017
- Deep Pyramid Convolutional Neural Networks for Text Categorization2017
- A Survey of Predictive Modeling on Imbalanced DomainsACM Computing Surveys, 2016
- Document Modeling with Gated Recurrent Neural Network for Sentiment Classification2015
- Convolutional Neural Networks for Sentence Classification2014
- Recursive regularization for large-scale classification with hierarchical and graphical dependencies2013
- Dimensionality Reduction by Learning an Invariant Mapping2006
- SMOTE: Synthetic Minority Over-sampling TechniqueJournal of Artificial Intelligence Research, 2002