Towards Boosting Many-to-Many Multilingual Machine Translation with Large Language Models
Preprint
Abstract
The training paradigm for machine translation has gradually shifted, from learning neural machine translation (NMT) models with extensive parallel corpora to instruction finetuning on multilingual large language models (LLMs) with high-quality translation pairs. In this paper, we focus on boosting many-to-many multilingual translation of LLMs with an emphasis on zero-shot translation directions. We demonstrate that prompt strategies adopted during finetuning are crucial to zero-shot translation and introduce a cross-lingual consistency regularization, XConST, to bridge the representation gap among different languages and improve zero-shot translation performance. XConST is not a new method, but a version of CrossConST (Gao et al., 2023a) adapted for translation instruction finetuning with LLMs. Experimental results on ALMA (Xu et al., 2023), Tower (Team, 2024), and LLaMA-2 (Touvron et al., 2023) show that our approach consistently improves translation performance. Our implementations are available at https://github.com/gpengzhi/CrossConST-LLM.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- Eliciting the Translation Ability of Large Language Models via Multilingual Finetuning with Translation InstructionsTransactions of the Association for Computational Linguistics, 2024
- Teaching Large Language Models to Translate with ComparisonProceedings of the AAAI Conference on Artificial Intelligence, 2024
- Learning Multilingual Sentence Representations with Cross-lingual Consistency Regularization2023
- Prompting PaLM for Translation: Assessing Strategies and Performance2023
- In-context Examples Selection for Machine Translation2023
- Improving Zero-shot Multilingual Neural Machine Translation by Leveraging Cross-lingual Consistency Regularization2023
- Few-shot Learning with Multilingual Generative Language Models2022
- Language Tags Matter for Zero-Shot Neural Machine Translation2021
- COMET: A Neural Framework for MT Evaluation2020
- Findings of the 2017 Conference on Machine Translation (WMT17)2017