Synthesizing Parallel Data of User-Generated Texts with Zero-Shot Neural Machine Translation
Authors:
Open Access
- 1 December 2020
- journal article
- research article
- Published by MIT Press in Transactions of the Association for Computational Linguistics
- Vol. 8, 710-725
Abstract
Neural machine translation (NMT) systems are usually trained on clean parallel data. They can perform very well for translating clean in-domain texts. However, as demonstrated by previous work, the translation quality significantly worsens when translating noisy texts, such as user-generated texts (UGT) from online social media. Given the lack of parallel data of UGT that can be used to train or adapt NMT systems, we synthesize parallel data of UGT, exploiting monolingual data of UGT through crosslingual language model pre-training and zero-shot NMT systems. This paper presents two different but complementary approaches: One alters given clean parallel data into UGT-like parallel data whereas the other generates translations from monolingual data of UGT. On the MTNT translation tasks, we show that our synthesized parallel data can lead to better NMT systems for UGT while making them more robust in translating texts from various domains and styles. Neural machine translation (NMT) systems are usually trained on clean parallel data. They can perform very well for translating clean in-domain texts. However, as demonstrated by previous work, the translation quality significantly worsens when translating noisy texts, such as user-generated texts (UGT) from online social media. Given the lack of parallel data of UGT that can be used to train or adapt NMT systems, we synthesize parallel data of UGT, exploiting monolingual data of UGT through crosslingual language model pre-training and zero-shot NMT systems. This paper presents two different but complementary approaches: One alters given clean parallel data into UGT-like parallel data whereas the other generates translations from monolingual data of UGT. On the MTNT translation tasks, we show that our synthesized parallel data can lead to better NMT systems for UGT while making them more robust in translating texts from various domains and styles. Neural machine translation (NMT) systems are usually trained on clean parallel data. They can perform very well for translating clean in-domain texts. However, as demonstrated by previous work, the translation quality significantly worsens when translating noisy texts, such as user-generated texts (UGT) from online social media. Given the lack of parallel data of UGT that can be used to train or adapt NMT systems, we synthesize parallel data of UGT, exploiting monolingual data of UGT through crosslingual language model pre-training and zero-shot NMT systems. This paper presents two different but complementary approaches: One alters given clean parallel data into UGT-like parallel data whereas the other generates translations from monolingual data of UGT. On the MTNT translation tasks, we show that our synthesized parallel data can lead to better NMT systems for UGT while making them more robust in translating texts from various domains and styles.Keywords
This publication has 15 references indexed in Scilit:
- Naver Labs Europe’s Systems for the WMT19 Machine Translation Robustness Task2019
- Phrase-Based & Neural Unsupervised Machine Translation2018
- MTNT: A Testbed for Machine Translation of Noisy Text2018
- Marian: Fast Neural Machine Translation in C++2018
- A Call for Clarity in Reporting BLEU Scores2018
- Style Transfer Through Back-Translation2018
- Neural Machine Translation of Rare Words with Subword Units2016
- Improving Neural Machine Translation Models with Monolingual Data2016
- chrF: character n-gram F-score for automatic MT evaluation2015
- Proceedings of the Tenth Workshop on Statistical Machine Translation2015