Debiasing Made State-of-the-art: Revisiting the Simple Seed-based Weak Supervision for Text Classification
Preprint
Abstract
Recent advances in weakly supervised text classification mostly focus on designing sophisticated methods to turn high-level human heuristics into quality pseudo-labels. In this paper, we revisit the seed matching-based method, which is arguably the simplest way to generate pseudo-labels, and show that its power was greatly underestimated. We show that the limited performance of seed matching is largely due to the label bias injected by the simple seed-match rule, which prevents the classifier from learning reliable confidence for selecting high-quality pseudo-labels. Interestingly, simply deleting the seed words present in the matched input texts can mitigate the label bias and help learn better confidence. Subsequently, the performance achieved by seed matching can be improved significantly, making it on par with or even better than the state-of-the-art. Furthermore, to handle the case when the seed words are not made known, we propose to simply delete the word tokens in the input text randomly with a high deletion ratio. Remarkably, seed matching equipped with this random deletion method can often achieve even better performance than that with seed deletion.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- Pre-trained Language Models Can be Fully Zero-Shot Learners2023
- LOPS: Learning Order Inspired Pseudo-Label Selection for Weakly Supervised Text Classification2022
- BFClass: A Backdoor-free Text Classification Framework2021
- ONION: A Simple and Effective Defense Against Textual Backdoor Attacks2021
- X-Class: Text Classification with Extremely Weak Supervision2021
- Weight Poisoning Attacks on Pretrained Models2020
- A Backdoor Attack Against LSTM-Based Text Classification SystemsIEEE Access, 2019
- PAWS: Paraphrase Adversaries from Word Scrambling2019
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2019