Bridging the domain gap in cross-lingual document classification
Preprint
Abstract
The scarcity of labeled training data often prohibits the internationalization of NLP models to multiple languages. Recent developments in cross-lingual understanding (XLU) has made progress in this area, trying to bridge the language barrier using language universal representations. However, even if the language problem was resolved, models trained in one language would not transfer to another language perfectly due to the natural domain drift across languages and cultures. We consider the setting of semi-supervised cross-lingual understanding, where labeled data is available in a source language (English), but only unlabeled data is available in the target language. We combine state-of-the-art cross-lingual methods with recently proposed methods for weakly supervised learning such as unsupervised pre-training and unsupervised data augmentation to simultaneously close both the language gap and the domain gap in XLU. We show that addressing the domain gap is crucial. We improve over strong baselines and achieve a new state-of-the-art for cross-lingual document classification.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and BeyondTransactions of the Association for Computational Linguistics, 2019
- BioBERT: a pre-trained biomedical language representation model for biomedical text miningBioinformatics, 2019
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2019
- An Embarrassingly Simple Approach for Transfer Learning from Pretrained Language Models2019
- XNLI: Evaluating Cross-lingual Sentence Representations2018
- Universal Language Model Fine-tuning for Text Classification2018
- Diverse Few-Shot Text Classification with Multiple Metrics2018
- Approaches to Cross-Domain Sentiment Analysis: A Systematic Literature ReviewIEEE Access, 2017
- Cross-lingual Distillation for Text Classification2017
- Learning Joint Multilingual Sentence Representations with Neural Machine Translation2017