Zero-Shot Learners for Natural Language Understanding via a Unified Multiple-Choice Perspective
Open Access
- 14 December 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 142829-142845
Abstract
Zero-shot learning is an approach where models generalize to unseen tasks without direct training on them. We introduce the Unified Multiple-Choice (UniMC) framework, which is format-independent, compatible with various formats, and applicable to tasks like text classification and sentiment analysis. Furthermore, we design a two-stage tuning method, initially training on multiple-choice formats to develop format-agnostic capabilities, and subsequently enabling direct predictions on unseen tasks for zero-shot learning. Our methodology avoids issues in large-scale models like FLAN, enhancing generalization and reducing parameters. In experiments, UniMC shows State-of-the-Art (SOTA) performance across out-of-domain and in-domain benchmarks, with only 235M parameters, far fewer than previous methods. Moreover, the UniMC-Chinese model excels beyond human performance on benchmarks like EPRSTMT and CHID-FC, underscoring its generalization capacity across languages. Additionally, ablation experiments demonstrate the effectiveness of our design.Keywords
Funding Information
- Fengshenbang Project [1]
This publication has 40 references indexed in Scilit:
- DREAM: A Challenge Data Set and Models for Dialogue-Based Reading ComprehensionTransactions of the Association for Computational Linguistics, 2019
- Constructing Datasets for Multi-hop Reading Comprehension Across DocumentsTransactions of the Association for Computational Linguistics, 2018
- Know What You Don’t Know: Unanswerable Questions for SQuAD2018
- Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering2018
- A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference2018
- Looking Beyond the Surface: A Challenge Set for Reading Comprehension over Multiple Sentences2018
- RACE: Large-scale ReAding Comprehension Dataset From Examinations2017
- A large annotated corpus for learning natural language inference2015
- The third PASCAL recognizing textual entailment challenge2007
- The PASCAL Recognising Textual Entailment Challenge2006