Neural Machine Translation Using Multiple Back-translation Generated by Sampling
Open Access
- 1 May 2020
- journal article
- research article
- Published by Japanese Society for Artificial Intelligence in Transactions of the Japanese Society for Artificial Intelligence
- Vol. 35 (3) , A-JA9_1-JA9_1
Abstract
Alarge-scaleparallelcorpusisindispensabletotrainencoder-decoderneuralmachinetranslation. Themethod of using synthetic parallel texts, called back-translation, in which target monolingual sentences are automatically translated into the source language, has been proven effective in improving the decoder. However, it does not necessarily help enhance the encoder. In this paper, we propose a method that enhances not only the decoder but also the encoder using target monolingual corpora by generating multiple source sentences via sampling-based sequence generation. The source sentences generated in that way increase their diversity and thus help make the encoder robust. Ourexperimentalresultsshowthatthetranslationqualitywasimprovedbyincreasingthenumberofsynthetic source sentences for each given target sentence. Even though the quality did not reach to the one that realized with a genuine parallel corpus comprising single human translations, our proposed method derived over 50% of the improvementbroughtbytheparallelcorpususingonlyitstargetside, i.e., monolingualdata. Moreover,theproposed samplingmethodresultedinfinaltranslationofhigherqualitythann-bestback-translation. Theseresultsindicatethat not only the quality of back-translation but also the diversity of synthetic source sentences is crucial.Keywords
This publication has 18 references indexed in Scilit:
- Recycling a Pre-trained BERT Encoder for Neural Machine Translation2019
- Understanding Back-Translation at Scale2018
- Enhancement of Encoder and Attention Using Target Monolingual Corpora in Neural Machine Translation2018
- Iterative Back-Translation for Neural Machine Translation2018
- Marian: Fast Neural Machine Translation in C++2018
- Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates2018
- Beam Search Strategies for Neural Machine Translation2017
- Using Target-side Monolingual Data for Neural Machine Translation through Multi-task Learning2017
- Copied Monolingual Data Improves Low-Resource Neural Machine Translation2017
- Semi-Supervised Learning for Neural Machine Translation2016