Transliterated Zero-Shot Domain Adaptation for Automatic Speech Recognition
Preprint
Abstract
The performance of automatic speech recognition models often degenerates on domains not covered by the training data. Domain adaptation can address this issue, assuming the availability of the target domain data in the target language. However, such assumption does not stand in many real-world applications. To make domain adaptation more applicable, we address the problem of zero-shot domain adaptation (ZSDA), where target domain data is unavailable in the target language. Instead, we transfer the target domain knowledge from another source language where the target domain data is more accessible. To do that, we first perform cross-lingual pre-training (XLPT) to share domain knowledge across languages, then use target language fine-tuning to build the final model. One challenge in this practice is that the pre-trained knowledge can be forgotten during fine-tuning, resulting in sub-optimal adaptation performance. To address this issue, we propose transliterated ZSDA to achieve consistent pre-training and fine-tuning labels, leading to maximum preservation of the pre-trained knowledge. Experimental results show that transliterated ZSDA relatively decreases the word error rate by 9.2% compared with a wav2vec 2.0 baseline. Moreover, transliterated ZSDA consistently outperforms self-supervised ZSDA and performs on par with supervised ZSDA, proving the superiority of transliteration-based pre-training labels.Keywords
All Related Versions
This publication has 20 references indexed in Scilit:
- Boosting Cross-Domain Speech Recognition With Self-SupervisionIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2023
- Deep transfer learning for automatic speech recognition: Towards better generalizationKnowledge-Based Systems, 2023
- Cross Corpus Speech Emotion Recognition using transfer learning and attention-based fusion of Wav2Vec2 and prosody featuresKnowledge-Based Systems, 2023
- Self-labeling with feature transfer for speech emotion recognitionKnowledge-Based Systems, 2022
- Cross-Lingual Self-training to Learn Multilingual Representation for Low-Resource Speech RecognitionCircuits, Systems, and Signal Processing, 2022
- Combining cross-modal knowledge transfer and semi-supervised learning for speech emotion recognitionKnowledge-Based Systems, 2021
- Adaptation Algorithms for Neural Network-Based Speech Recognition: An OverviewIEEE Open Journal of Signal Processing, 2020
- A Teacher-Student Learning Approach for Unsupervised Domain Adaptation of Sequence-Trained ASR Models2018
- AISHELL-1: An open-source Mandarin speech corpus and a speech recognition baseline2017
- An unsupervised deep domain adaptation approach for robust speech recognitionNeurocomputing, 2017