Incorporating Discrete Translation Lexicons into Neural Machine Translation
Preprint
Abstract
Neural machine translation (NMT) often makes mistakes in translating low-frequency content words that are essential to understanding the meaning of the sentence. We propose a method to alleviate this problem by augmenting NMT systems with discrete translation lexicons that efficiently encode translations of these low-frequency words. We describe a method to calculate the lexicon probability of the next word in the translation candidate by using the attention vector of the NMT model to select which source word lexical probabilities the model should focus on. We test two methods to combine this probability with the standard NMT probability: (1) using it as a bias, and (2) linear interpolation. Experiments on two corpora show an improvement of 2.0-2.3 BLEU and 0.13-0.44 NIST score, and faster convergence time.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- Achieving Open Vocabulary Neural Machine Translation with Hybrid Word-Character Models2016
- Pointing the Unknown Words2016
- Incorporating Copying Mechanism in Sequence-to-Sequence Learning2016
- A Character-level Decoder without Explicit Segmentation for Neural Machine Translation2016
- Improving Neural Machine Translation Models with Monolingual Data2016
- Addressing the Rare Word Problem in Neural Machine Translation2015
- On Using Very Large Target Vocabulary for Neural Machine Translation2015
- Hierarchical Phrase-Based TranslationComputational Linguistics, 2007
- Learning to Forget: Continual Prediction with LSTMNeural Computation, 2000