On Instruction-Finetuning Neural Machine Translation Models
Preprint
Abstract
In this work, we introduce instruction finetuning for Neural Machine Translation (NMT) models, which distills instruction following capabilities from Large Language Models (LLMs) into orders-of-magnitude smaller NMT models. Our instruction-finetuning recipe for NMT models enables customization of translations for a limited but disparate set of translation-specific tasks. We show that NMT models are capable of following multiple instructions simultaneously and demonstrate capabilities of zero-shot composition of instructions. We also show that through instruction finetuning, traditionally disparate tasks such as formality-controlled machine translation, multi-domain adaptation as well as multi-modal translations can be tackled jointly by a single instruction finetuned NMT model, at a performance level comparable to LLMs such as GPT-3.5-Turbo. To the best of our knowledge, our work is among the first to demonstrate the instruction-following capabilities of traditional NMT models, which allows for faster, cheaper and more efficient serving of customized translations.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- GATE: A Challenge Set for Gender-Ambiguous Translation Examples2023
- Self-Instruct: Aligning Language Models with Self-Generated Instructions2023
- Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor2023
- Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A SurveyJournal of Artificial Intelligence Research, 2022
- Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks2022
- CoCoA-MT: A Dataset and Benchmark for Contrastive Controlled MT with Application to Formality2022
- Controlling Translation Formality Using Pre-trained Multilingual Language Models2022
- Cross-Task Generalization via Natural Language Crowdsourcing Instructions2022
- Controlling Machine Translation for Multiple Attributes with Additive Interventions2021
- COMET: A Neural Framework for MT Evaluation2020