Joint Speech Transcription and Translation: Pseudo-Labeling with Out-of-Distribution Data
Preprint
Abstract
Self-training has been shown to be helpful in addressing data scarcity for many domains, including vision, speech, and language. Specifically, self-training, or pseudo-labeling, labels unsupervised data and adds that to the training pool. In this work, we investigate and use pseudo-labeling for a recently proposed novel setup: joint transcription and translation of speech, which suffers from an absence of sufficient data resources. We show that under such data-deficient circumstances, the unlabeled data can significantly vary in domain from the supervised data, which results in pseudo-label quality degradation. We investigate two categories of remedies that require no additional supervision and target the domain mismatch: pseudo-label filtering and data augmentation. We show that pseudo-label analysis and processing as such results in additional gains on top of the vanilla pseudo-labeling setup resulting in total improvements of up to 0.6% absolute WER and 2.2 BLEU points.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- Self-Training and Pre-Training are Complementary for Speech Recognition2021
- MuST-C: A multilingual corpus for end-to-end speech translationComputer Speech & Language, 2021
- Data Augmentation by Concatenation for Low-Resource Translation: A Mystery and a Solution2021
- Consistent Transcription and Translation of SpeechTransactions of the Association for Computational Linguistics, 2020
- Self-Training for End-to-End Speech Recognition2020
- Transformers: State-of-the-Art Natural Language Processing2020
- Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and BeyondTransactions of the Association for Computational Linguistics, 2019
- SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing2018
- Connectionist temporal classification2006
- Probability of error of some adaptive pattern-recognition machinesIEEE Transactions on Information Theory, 1965