Adversarial Domain Adaptation for Machine Reading Comprehension
Preprint
Abstract
In this paper, we focus on unsupervised domain adaptation for Machine Reading Comprehension (MRC), where the source domain has a large amount of labeled data, while only unlabeled passages are available in the target domain. To this end, we propose an Adversarial Domain Adaptation framework (AdaMRC), where ($i$) pseudo questions are first generated for unlabeled passages in the target domain, and then ($ii$) a domain classifier is incorporated into an MRC model to predict which domain a given passage-question pair comes from. The classifier and the passage-question encoder are jointly trained using adversarial learning to enforce domain-invariant representation learning. Comprehensive evaluations demonstrate that our approach ($i$) is generalizable to different MRC models and datasets, ($ii$) can be combined with pre-trained large-scale language models (such as ELMo and BERT), and ($iii$) can be extended to semi-supervised learning.Keywords
All Related Versions
This publication has 25 references indexed in Scilit:
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2019
- Multi-task Learning with Sample Re-weighting for Machine Reading Comprehension2019
- Adversarial Deep Averaging Networks for Cross-Lingual Sentiment ClassificationTransactions of the Association for Computational Linguistics, 2018
- Maximum Classifier Discrepancy for Unsupervised Domain Adaptation2018
- Answer-focused and Position-aware Neural Question Generation2018
- Self-Training for Jointly Learning to Ask and Answer Questions2018
- Two-Stage Synthesis Networks for Transfer Learning in Machine Comprehension2017
- SQuAD: 100,000+ Questions for Machine Comprehension of Text2016
- Pointing the Unknown Words2016
- Geodesic flow kernel for unsupervised domain adaptation2012