Iterative Training of Unsupervised Neural and Statistical Machine Translation Systems
Authors:
Open Access
- 1 June 2020
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Asian and Low-Resource Language Information Processing
- Vol. 19 (5) , 1-21
Abstract
Recent work achieved remarkable results in training neural machine translation (NMT) systems in a fully unsupervised way, with new and dedicated architectures that only rely on monolingual corpora. However, previous work also showed that unsupervised statistical machine translation (USMT) performs better than unsupervised NMT (UNMT), especially for distant language pairs. To take advantage of the superiority of USMT over UNMT, and considering that SMT suffers from well-known limitations overcome by NMT, we propose to define UNMT as NMT trained with the supervision of synthetic parallel data generated by USMT. This way we can exploit USMT up to its limits while ultimately relying on full-fledged NMT models to generate translations. We show significant improvements in translation quality over previous work and also that further improvements can be obtained by alternatively and iteratively training USMT and UNMT. Without the need of a dedicated architecture for UNMT, our simple approach can straightforwardly benefit from any recent and future advances in supervised NMT. Our systems achieve a new state-of-the-art for unsupervised machine translation in all of our six translation tasks for five diverse language pairs, surpassing even supervised SMT or NMT in some tasks. Furthermore, our analysis shows how crucial the comparability between the monolingual corpora used for unsupervised training is in improving translation quality.Keywords
This publication has 19 references indexed in Scilit:
- Phrase Table Induction Using Monolingual Data for Low-Resource Statistical Machine TranslationACM Transactions on Asian and Low-Resource Language Information Processing, 2018
- Marian: Fast Neural Machine Translation in C++2018
- A Call for Clarity in Reporting BLEU Scores2018
- Enriching Word Vectors with Subword InformationTransactions of the Association for Computational Linguistics, 2017
- Joint Training for Pivot-based Neural Machine Translation2017
- Introduction of the Asian Language Treebank2016
- Improving Neural Machine Translation Models with Monolingual Data2016
- Improving fast_align by Reordering2015
- Unsupervised Sparse Vector Densification for Short Text Similarity2015
- Learning Translation Models from Monolingual Continuous Representations2015