End-end Speech-to-Text Translation with Modality Agnostic Meta-Learning
Authors:
- 1 May 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 7904-7908
Abstract
Collecting large amounts of data to train end-to-end Speech Translation (ST) models is more difficult compared to the ASR and MT tasks. Previous studies have proposed the use of transfer learning approaches to overcome the above difficulty. These approaches benefit from weakly supervised training data, such as ASR speech-to-transcript or MT text-to-text translation pairs. However, the parameters in these models are updated independently of each task, which may lead to sub-optimal solutions. In this work, we adopt a meta-learning algorithm to train a modality agnostic multi-task model that transfers knowledge from source tasks=ASR+MT to target task=ST where the ST task severely lacks data. In the meta-learning phase, parameters are updated in such a way that they act as a good ini-tialization point for the target ST task. We evaluate the proposed meta-learning approach for ST tasks on English-German (En-De) and English-French (En-Fr) language pairs from the Multilingual Speech Translation Corpus (MuST-C). Our method outperforms the previous transfer learning approaches and sets new state-of-the-art results for En-De and En-Fr ST tasks by obtaining 9.18, and 11.76 BLEU point improvements, respectively.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- Leveraging Weakly Supervised Data to Improve End-to-end Speech-to-text Translation2019
- End-to-End Automatic Speech Translation of Audiobooks2018
- The Spoken Wikipedia Corpus collection: Harvesting, alignment and an application to hyperlisteningLanguage Resources and Evaluation, 2018
- Universal Neural Machine Translation for Extremely Low Resource Languages2018
- Tied Multitask Learning for Neural Speech Translation2018
- Sequence-to-Sequence Models Can Directly Translate Foreign Speech2017
- Neural Machine Translation of Rare Words with Subword Units2016
- An Attentional Model for Speech Translation Without Transcription2016
- Speech recognition with deep recurrent neural networks2013
- Speech translation: coupling of recognition and translation1999