Breaking the Curse of Multilinguality with Cross-lingual Expert Language Models
Authors:
- Terra BlevinsTBTerra Blevins
- Suchin GururanganSGSuchin Gururangan
- Margaret LiMLMargaret Li
- Hila GonenHGHila Gonen
- Luke ZettlemoyerLZLuke Zettlemoyer
Open Access
- 1 January 2024
- conference paper
- Published by Association for Computational Linguistics (ACL)
- p. 10822-10837
Abstract
Terra Blevins, Tomasz Limisiewicz, Suchin Gururangan, Margaret Li, Hila Gonen, Noah A. Smith, Luke Zettlemoyer. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- Overcoming Catastrophic Forgetting in Massively Multilingual Continual Learning2023
- Quality at a Glance: An Audit of Web-Crawled Multilingual DatasetsTransactions of the Association for Computational Linguistics, 2022
- Few-shot Learning with Multilingual Generative Language Models2022
- Small Data? No Problem! Exploring the Viability of Pretrained Multilingual Language Models for Low-resourced Languages2021
- Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inference2021
- mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer2021
- MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer2020
- Unsupervised Cross-lingual Representation Learning at Scale2020
- Are All Languages Created Equal in Multilingual BERT?2020
- Adaptive Mixtures of Local ExpertsNeural Computation, 1991