Noise-Aware Extended U-Net With Split Encoder and Feature Refinement Module for Robust Speaker Verification in Noisy Environments
Open Access
- 25 July 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 111673-111682
Abstract
Speech data gathered from real-world environments typically contain noise, a significant element that undermines the performance of deep neural network-based speaker verification (SV) systems. To mitigate performance degradation due to noise and construct noise-robust SV systems, several researchers have employed frameworks that integrate speech enhancement (SE) and SV systems. We have proposed the extended U-Net (ExU-Net) that has achieved state-of-the-art performance in SV in noisy environments by jointly training SE and SV systems. In the SE field, some studies have shown that recognizing noise components within speech can improve the system’s performance. Inspired by these approaches, we propose a noise-aware ExU-Net (NA-ExU-Net) that acknowledges noise information in the SE process based on the ExU-Net architecture. The proposed system comprises a Split Encoder and a feature refinement module (FRM). The Split Encoder handles the speech and noise separately by dividing the encoder blocks, whereas FRM is designed to inhibit the propagation of irrelevant data via skip connections. To validate the effectiveness of our proposed framework in noisy conditions, we evaluated the models on the VoxCeleb1 test set with added noise from the MUSAN corpus. The experimental results demonstrate that NA-ExU-Net outperforms ExU-Net and other baseline systems under all evaluation conditions. Furthermore, evaluations in out-of-domain noise environments indicate that NA-ExU-Net significantly surpasses existing frameworks, highlighting its robustness and generalization capabilities. The codes utilized in our experiments can be accessed at https://github.com/chan-yeong0519/NA-ExU-Net.Keywords
Funding Information
- National Research Foundation of Korea (NRF) grant funded by the Korea government (2023R1A2C1005744)
- 2023 Advanced Facility Fund of the University of Seoul
This publication has 40 references indexed in Scilit:
- MFA: TDNN with Multi-Scale Frequency-Channel Attention for Text-Independent Speaker Verification with Short Utterances2022
- ResNeXt and Res2Net Structures for Speaker Verification2021
- Within-Sample Variability-Invariant Loss for Robust Speaker Recognition Under Noisy Environments2020
- Improving Deep CNN Networks with Long Temporal Context for Text-Independent Speaker Verification2020
- ArcFace: Additive Angular Margin Loss for Deep Face Recognition2019
- NAS-Unet: Neural Architecture Search for Medical Image SegmentationIEEE Access, 2019
- Squeeze-and-Excitation Networks2018
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Speaker and Session Variability in GMM-Based Speaker VerificationIEEE Transactions on Audio, Speech, and Language Processing, 2007
- Support vector machines using GMM supervectors for speaker verificationIEEE Signal Processing Letters, 2006