Task Aware Multi-Task Learning for Speech to Text Tasks
Authors:
- 6 June 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 7723-7727
Abstract
In general, the direct Speech-to-text translation (ST) is jointly trained with Automatic Speech Recognition (ASR), and Machine Translation (MT) tasks. However, the issues with the current joint learning strategies inhibit the knowledge transfer across these tasks. We propose a task modulation network which allows the model to learn task specific features, while learning the shared features simultaneously. This proposed approach removes the need for separate finetuning step resulting in a single model which performs all these tasks. This single model achieves a performance of 28.64 BLEU score on ST MuST-C English-German, WER of 11.61% on ASR TEDLium v3, 23.35 BLEU score on MT WMT’15 English-German task. This sets a new state-of-the-art performance (SOTA) on the ST task while outperforming the existing end-to-end ASR systems.Keywords
This publication has 15 references indexed in Scilit:
- TED-LIUM 3: Twice as Much Data and Corpus Repartition for Experiments on Speaker Adaptation2018
- End-to-End Automatic Speech Translation of Audiobooks2018
- Multilingual Speech Recognition with a Single End-to-End Model2018
- A Call for Clarity in Reporting BLEU Scores2018
- Universal Neural Machine Translation for Extremely Low Resource Languages2018
- Neural Machine Translation of Rare Words with Subword Units2016
- Librispeech: An ASR corpus based on public domain audio books2015
- Effective Approaches to Attention-based Neural Machine Translation2015
- Representation Learning Using Multi-Task Deep Neural Networks for Semantic Classification and Information Retrieval2015
- Multitask LearningMachine Learning, 1997