Multilingual Speech Recognition with a Single End-to-End Model
Authors:
- 1 April 2018
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 4904-4908
Abstract
Training a conventional automatic speech recognition (ASR) system to support multiple languages is challenging because the sub-word unit, lexicon and word inventories are typically language specific. In contrast, sequence-to-sequence models are well suited for multilingual ASR because they encapsulate an acoustic, pronunciation and language model jointly in a single network. In this work we present a single sequence-to-sequence ASR model trained on 9 different Indian languages, which have very little overlap in their scripts. Specifically, we take a union of language-specific grapheme sets and train a grapheme-based sequence-to-sequence model jointly on data from all languages. We find that this model, which is not explicitly given any information about language identity, improves recognition performance by 21% relative compared to analogous sequence-to-sequence models trained on each language individually. By modifying the model to accept a language identifier as an additional input feature, we further improve performance by an additional 7% relative and eliminate confusion between different languages.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Towards Better Decoding and Language Model Integration in Sequence to Sequence Models2017
- Automatic speech recognition for under-resourced languages: A surveySpeech Communication, 2014
- Language-dependent state clustering for multilingual acoustic modellingSpeech Communication, 2007
- Language-independent and language-adaptive acoustic modeling for speech recognitionSpeech Communication, 2001
- Bidirectional recurrent neural networksIEEE Transactions on Signal Processing, 1997
- Long Short-Term MemoryNeural Computation, 1997
- Fast bootstrapping of LVCSR systems with multilingual phoneme sets1997
- Multitask LearningMachine Learning, 1997