Abstract
Speech enhancement techniques based on deep learning have demonstrated substantial improvements over traditional signal processing methods, particularly under complex non-stationary noise conditions. In this paper, we propose a novel lightweight speech enhancement model, ATF-MUSE, which is designed to more effectively model the complex interactions between temporal- and frequency-domain features, while dynamically adjusting attention weights to enhance adaptability to varying noise environments. Specifically, we introduce an ATF-Transformer module that alternately models dependencies along the temporal and frequency dimensions, thereby capturing both the global spectral patterns and long-term temporal relationships of speech signals. Moreover, we propose an Adaptive Multi-Head Attention (AMHA) mechanism that dynamically generates feature-dependent attention weights, significantly improving the model’s robustness against non-stationary noise. We evaluate the proposed ATF-MUSE model on the widely used VoiceBank-DEMAND dataset, and experimental results demonstrate that our method outperforms several state-of-the-art approaches across multiple standard evaluation metrics, including PESQ, MOS, and STOI.
Funding Information
  • National Science and Technology Major Project (2025ZD0619602)