Universal Robust Speech Adaptation for Cross-Domain Speech Recognition and Enhancement
- 9 February 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech and Language Processing
- Vol. 34, 1229-1242
Abstract
Pre-trained models for automatic speech recognition (ASR) and speech enhancement (SE) have exhibited remarkable capabilities under matched noise and channel conditions. However, these models often suffer from severe performance degradation when confronted with domain shifts, particularly in the presence of unseen noise and channel distortions. In view of this, we in this paper present URSA-GAN, a unified and domain-aware generative framework specifically designed to mitigate mismatches in both noise and channel conditions. URSA-GAN leverages a dual-embedding architecture that consists of a noise encoder and a channel encoder, each pre-trained with limited in-domain data to capture domain-relevant representations. These embeddings condition a GAN-based speech generator, facilitating the synthesis of speech that is acoustically aligned with the target domain while preserving phonetic content. To enhance generalization further, we propose dynamic stochastic perturbation, a novel regularization technique that introduces controlled variability into the embeddings during generation, promoting robustness to unseen domains. Empirical results demonstrate that URSA-GAN effectively reduces character error rates in ASR and improves perceptual metrics in SE across diverse noisy and mismatched channel scenarios. Notably, evaluations on compound test conditions with both channel and noise degradations confirm the generalization ability of URSA-GAN, yielding relative improvements of 16.16% in ASR performance and 15.58% in SE metrics.Keywords
All Related Versions
Funding Information
- National Science and Technology Council (NSTC 112-2221-E-001-009-MY3)
This publication has 54 references indexed in Scilit:
- CTSE-Net: Resource-efficient convolutional and TF-transformer network for speech enhancementKnowledge-Based Systems, 2025
- MA-Net: Resource-efficient multi-attentional network for end-to-end speech enhancementNeurocomputing, 2024
- Reproducing Whisper-Style Training Using An Open-Source Toolkit And Publicly Available Data2023
- Contrastive Learning for Unpaired Image-to-Image Translation2020
- Channel Mismatch Speaker Verification Based on Deep Learning and PLDAJournal of Physics: Conference Series, 2020
- Generative Adversarial Networks Based Data Augmentation for Noise Robust Speech Recognition2018
- Convolutional-Recurrent Neural Networks for Speech Enhancement2018
- Adversarial Discriminative Domain Adaptation2017
- Transfer learning for speech and language processing2015
- Perceptual evaluation of speech quality (PESQ)-a new method for speech quality assessment of telephone networks and codecs2002