Multilingual End-to-End Speech Translation
Authors:
- 1 December 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 570-577
Abstract
In this paper, we propose a simple yet effective framework for multilingual end-to-end speech translation (ST), in which speech utterances in source languages are directly translated to the desired target languages with a universal sequence-to-sequence architecture. While multilingual models have shown to be useful for automatic speech recognition (ASR) and machine translation (MT), this is the first time they are applied to the end-to-end ST problem. We show the effectiveness of multilingual end-to-end ST in two scenarios: one-to-many and many-to-many translations with publicly available data. We experimentally confirm that multilingual end-to-end ST models significantly outperform bilingual ones in both scenarios. The generalization of multilingual training is also evaluated in a transfer learning scenario to a very low-resource language pair. All of our codes and the database are publicly available to encourage further research in this emergent multilingual ST topic 1 1 Available at https://github.com/espnet/espnet..Keywords
All Related Versions
This publication has 27 references indexed in Scilit:
- Listen, attend and spell: A neural network for large vocabulary conversational speech recognition2016
- Neural Machine Translation of Rare Words with Subword Units2016
- Improving Neural Machine Translation Models with Monolingual Data2016
- Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism2016
- Multi-Task Learning for Multiple Language Translation2015
- Multilingual deep neural network based acoustic modeling for rapid language adaptation2014
- The language-independent bottleneck features2012
- Connectionist temporal classification2006
- Speech translation: coupling of recognition and translation1999
- Long Short-Term MemoryNeural Computation, 1997