Extremely low-resource neural machine translation for Asian languages
Open Access
- 1 December 2020
- journal article
- research article
- Published by Springer Nature in Machine Translation
- Vol. 34 (4) , 347-382
Abstract
This paper presents a set of effective approaches to handle extremely low-resource language pairs for self-attention based neural machine translation (NMT) focusing on English and four Asian languages. Starting from an initial set of parallel sentences used to train bilingual baseline models, we introduce additional monolingual corpora and data processing techniques to improve translation quality. We describe a series of best practices and empirically validate the methods through an evaluation conducted on eight translation directions, based on state-of-the-art NMT approaches such as hyper-parameter search, data augmentation with forward and backward translation in combination with tags and noise, as well as joint multilingual training. Experiments show that the commonly used default architecture of self-attention NMT models does not reach the best results, validating previous work on the importance of hyper-parameter tuning. Additionally, empirical results indicate the amount of synthetic data required to efficiently increase the parameters of the models leading to the best translation quality measured by automatic metrics. We show that the best NMT models trained on large amount of tagged back-translations outperform three other synthetic data generation approaches. Finally, comparison with statistical machine translation (SMT) indicates that extremely low-resource NMT requires a large amount of synthetic parallel data obtained with back-translation in order to close the performance gap with the preceding SMT approach.Keywords
This publication has 37 references indexed in Scilit:
- Neural Machine Translation of Rare Words with Subword Units2016
- Improving Neural Machine Translation Models with Monolingual Data2016
- Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism2016
- chrF: character n-gram F-score for automatic MT evaluation2015
- On the Properties of Neural Machine Translation: Encoder–Decoder Approaches2014
- Phrase-Based Statistical Machine Translation2002
- BLEU2001
- Improved statistical alignment models2000
- Recursive hetero-associative memories for translationLecture Notes in Computer Science, 1997
- Aligning sentences in parallel corpora1991