Nearest Neighbor Machine Translation
Preprint
Abstract
We introduce $k$-nearest-neighbor machine translation ($k$NN-MT), which predicts tokens with a nearest neighbor classifier over a large datastore of cached examples, using representations from a neural translation model for similarity search. This approach requires no additional training and scales to give the decoder direct access to billions of examples at test time, resulting in a highly expressive model that consistently improves performance across many settings. Simply adding nearest neighbor search improves a state-of-the-art German-English translation model by 1.5 BLEU. $k$NN-MT allows a single model to be adapted to diverse domains by using a domain-specific datastore, improving results by an average of 9.2 BLEU over zero-shot transfer, and achieving new state-of-the-art results -- without training on these domains. A massively multilingual model can also be specialized for particular language pairs, with improvements of 3 BLEU for translating from English into German and Chinese. Qualitatively, $k$NN-MT is easily interpretable; it combines source and target context to retrieve highly relevant examples.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Augmenting Transformers with KNN-Based Composite Memory for DialogTransactions of the Association for Computational Linguistics, 2021
- CCMatrix: Mining Billions of High-Quality Parallel Sentences on the Web2021
- Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation2020
- Billion-Scale Similarity Search with GPUsIEEE Transactions on Big Data, 2019
- Findings of the 2019 Conference on Machine Translation (WMT19)2019
- Findings of the 2018 Conference on Machine Translation (WMT18)2018
- Retrieve and Refine: Improved Sequence Generation Models For Dialogue2018
- Example-based machine translation using efficient sentence retrieval based on edit-distanceACM Transactions on Asian Language Information Processing, 2005