Leveraging Domain Knowledge at Inference Time for LLM Translation: Retrieval versus Generation
Authors:
- Bryan LiBLBryan Li
- Jiaming LuoJLJiaming Luo
- Eleftheria BriakouEBEleftheria Briakou
- Colin CherryCCColin Cherry
Preprint
Abstract
While large language models (LLMs) have been increasingly adopted for machine translation (MT), their performance for specialist domains such as medicine and law remains an open challenge. Prior work has shown that LLMs can be domain-adapted at test-time by retrieving targeted few-shot demonstrations or terminologies for inclusion in the prompt. Meanwhile, for general-purpose LLM MT, recent studies have found some success in generating similarly useful domain knowledge from an LLM itself, prior to translation. Our work studies domain-adapted MT with LLMs through a careful prompting setup, finding that demonstrations consistently outperform terminology, and retrieval consistently outperforms generation. We find that generating demonstrations with weaker models can close the gap with larger model's zero-shot performance. Given the effectiveness of demonstrations, we perform detailed analyses to understand their value. We find that domain-specificity is particularly important, and that the popular multi-domain benchmark is testing adaptation to a particular writing style more so than to a specific domain.Keywords
All Related Versions
This publication has 14 references indexed in Scilit:
- Exploring Human-Like Translation Strategy with Large Language ModelsTransactions of the Association for Computational Linguistics, 2024
- Chain-of-Dictionary Prompting Elicits Translation in Large Language Models2024
- Narrowing the Gap between Zero- and Few-shot Machine Translation by Matching Styles2024
- Findings of the WMT 2023 Shared Task on Machine Translation with Terminologies2023
- Prompting PaLM for Translation: Assessing Strategies and Performance2023
- In-context Examples Selection for Machine Translation2023
- A Template-based Method for Constrained Neural Machine Translation2022
- Unsupervised Domain Clusters in Pretrained Language Models2020
- Purest ever example-based machine translation: Detailed presentation and assessmentMachine Translation, 2006
- Review Article: Example-based Machine TranslationMachine Translation, 1999