Semi-supervised acoustic model training for speech with code-switching
Preprint
Abstract
In the FAME! project, we aim to develop an automatic speech recognition (ASR) system for Frisian-Dutch code-switching (CS) speech extracted from the archives of a local broadcaster with the ultimate goal of building a spoken document retrieval system. Unlike Dutch, Frisian is a low-resourced language with a very limited amount of manually annotated speech data. In this paper, we describe several automatic annotation approaches to enable using of a large amount of raw bilingual broadcast data for acoustic model training in a semi-supervised setting. Previously, it has been shown that the best-performing ASR system is obtained by two-stage multilingual deep neural network (DNN) training using 11 hours of manually annotated CS speech (reference) data together with speech data from other high-resourced languages. We compare the quality of transcriptions provided by this bilingual ASR system with several other approaches that use a language recognition system for assigning language labels to raw speech segments at the front-end and using monolingual ASR resources for transcription. We further investigate automatic annotation of the speakers appearing in the raw broadcast data by first labeling with (pseudo) speaker tags using a speaker diarization system and then linking to the known speakers appearing in the reference data using a speaker recognition system. These speaker labels are essential for speaker-adaptive training in the proposed setting. We train acoustic models using the manually and automatically annotated data and run recognition experiments on the development and test data of the FAME! speech corpus to quantify the quality of the automatic annotations. The ASR and CS detection results demonstrate the potential of using automatic language and speaker tagging in semi-supervised bilingual acoustic model training.Keywords
All Related Versions
This publication has 28 references indexed in Scilit:
- A study of speaker clustering for speaker attribution in large telephone conversation datasetsComputer Speech & Language, 2016
- Lithuanian Broadcast Speech Transcription Using Semi-supervised Acoustic Model TrainingProcedia Computer Science, 2016
- Phonetisaurus: Exploring grapheme-to-phoneme conversion with joint n-gram models in the WFST frameworkNatural Language Engineering, 2015
- Mandarin–English code-switching speech corpus in South-East Asia: SEAMELanguage Resources and Evaluation, 2015
- Automatic speech recognition for under-resourced languages: A surveySpeech Communication, 2014
- Speaker-adaptive speech recognition using speaker diarization for improved transcription of large spoken archivesSpeech Communication, 2013
- Using out-of-language data to improve an under-resourced speech recognizerSpeech Communication, 2013
- Unsupervised training and directed manual transcription for LVCSRSpeech Communication, 2010
- Lightly supervised and unsupervised acoustic model trainingComputer Speech & Language, 2002
- Maximum likelihood linear transformations for HMM-based speech recognitionComputer Speech & Language, 1998