Multilingual and cross-lingual document classification: A meta-learning approach
Preprint
Abstract
The great majority of languages in the world are considered under-resourced for the successful application of deep learning methods. In this work, we propose a meta-learning approach to document classification in limited-resource setting and demonstrate its effectiveness in two different settings: few-shot, cross-lingual adaptation to previously unseen languages; and multilingual joint training when limited target-language data is available during training. We conduct a systematic comparison of several meta-learning methods, investigate multiple settings in terms of data availability and show that meta-learning thrives in settings with a heterogeneous task distribution. We propose a simple, yet effective adjustment to existing meta-learning methods which allows for better and more stable learning, and set a new state of the art on several languages while performing on-par on others, using only a small amount of labeled data.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- Gradient Centralization: A New Optimization Technique for Deep Neural Networks2020
- Learning to Few-Shot Learn Across Diverse Natural Language Classification Tasks2020
- Zero-Shot Cross-Lingual Transfer with Meta Learning2020
- Unsupervised Cross-lingual Representation Learning at Scale2020
- MLQA: Evaluating Cross-lingual Extractive Question Answering2020
- Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and BeyondTransactions of the Association for Computational Linguistics, 2019
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2019
- Investigating Meta-Learning Algorithms for Low-Resource Natural Language Understanding Tasks2019
- Meta Relational Learning for Few-Shot Link Prediction in Knowledge Graphs2019
- XNLI: Evaluating Cross-lingual Sentence Representations2018