Multilingual NMT with a language-independent attention bridge
Authors:
Preprint
Abstract
In this paper, we propose a multilingual encoder-decoder architecture capable of obtaining multilingual sentence representations by means of incorporating an intermediate {\em attention bridge} that is shared across all languages. That is, we train the model with language-specific encoders and decoders that are connected via self-attention with a shared layer that we call attention bridge. This layer exploits the semantics from each language for performing translation and develops into a language-independent meaning representation that can efficiently be used for transfer learning. We present a new framework for the efficient development of multilingual NMT using this model and scheduled training. We have tested the approach in a systematic way with a multi-parallel data set. We show that the model achieves substantial improvements over strong bilingual models and that it also works well for zero-shot translation, which demonstrates its ability of abstraction and transfer learning.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- Are BLEU and Meaning Representation in Opposition?2018
- What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties2018
- A neural interlingua for multilingual machine translation2018
- A Call for Clarity in Reporting BLEU Scores2018
- Universal Neural Machine Translation for Extremely Low Resource Languages2018
- Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot TranslationTransactions of the Association for Computational Linguistics, 2017
- A Teacher-Student Framework for Zero-Resource Neural Machine Translation2017
- OpenNMT: Open-Source Toolkit for Neural Machine Translation2017
- Zero-Resource Translation with Multi-Lingual Neural Machine Translation2016
- Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism2016