Hyung-Min Park

In severe noise conditions, employing a speech enhancement (SE) model as a front-end serves as a computationally efficient strategy for robust automatic speech recognition (ASR), offering a practical alternative to the costly fine-tuning of largescale ASR systems. However, improvements in human perceptual quality do not necessarily guarantee enhanced machine recognition accuracy, as aggressive noise suppression often introduces distortions and artifacts that obscure fine-grained spectral details and increase word error rates (WERs). To mitigate the discrepancy, we present the Dynamic Distortion Control (DDC) loss, a unified training objective designed to bridge the gap between perceptual fidelity and recognition robustness. Integrated into a time-frequency Transformer architecture, the presented loss addresses the distortion-robustness discrepancy. Experimental results on a LibriSpeech dataset corrupted by noise from DNS Challenge demonstrate the effectiveness of the DDC loss on both perceptual quality and recognition accuracy across diverse noise conditions.

This paper presents an efficient speech enhancement (SE) approach that reuses a processing block repeatedly instead of conventional stacking. Rather than increasing the number of blocks for learning deep latent representations, repeating a single block leads to progressive refinement while reducing parameter redundancy. We also minimize domain transformation by keeping an encoder and decoder shallow and reusing a single sequence modeling block. Experimental results show that the number of processing stages is more critical to performance than the number of blocks with different weights. Also, we observed that the proposed method gradually refines a noisy input within a single block. Furthermore, with the block reuse method, we demonstrate that deepening the encoder and decoder can be redundant for learning deep complex representation. Therefore, the experimental results confirm that the proposed block reusing enables progressive learning and provides an efficient alternative for SE.

In general, multi-channel source separation has utilized inter-microphone phase differences (IPDs) concatenated with magnitude information in time-frequency domain, or real and imaginary components stacked along the channel axis. However, the spatial information of a sound source is fundamentally contained in the “differences” between microphones, specifically in the correlation between them, while the power of each microphone also provides valuable information about the source spectrum, which is why the magnitude is also included. Therefore, we propose a network that directly leverages a correlation input with phase transform (PHAT)-$\beta$ to estimate the separation filter. In addition, the proposed TF-CorrNet processes the features alternately across time and frequency axes as a dual-path strategy in terms of spatial information. Furthermore, we add a spectral module to model source-related direct time-frequency patterns for improved speech separation. Experimental results demonstrate that the proposed TF-CorrNet effectively separates the speech sounds, showing high performance with a low computational cost in the LibriCSS dataset.

With the progress of artificial intelligence (AI) technology, home appliances are becoming more advanced to enhance our quality of life. Many smart devices support speech interfaces, including voice commands and user location tracking. However, robotic vacuum cleaners generate strong ego-noise that distorts microphone signals, making it difficult to estimate the user’s location. To solve this problem, we propose a real-time sound source localization (SSL) system for a robotic vacuum cleaner equipped with a microphone array. We design a system that consists of speech enhancement, voice activity detection (VAD), and SSL modules. The speech enhancement module includes TRU-Net-Light, which has lower computation and similar speech enhancement performance to TRU-Net. The TRU-Net-Light reduces the number of channels to reduce the model size and applies a frequency-axis multi-head self-attention to boost representational capacity. The finite state machine-based VAD is designed to detect voice active periods using the output of a speech enhancement module. Furthermore, we present a mask-weighted difference correlation vector and the singular value decomposition with smoother coherence transform (DSVD-SCOT) that achieves robust localization performance in severely noisy environments. In the experimented robotic vacuum cleaner, the localization accuracy of the SSL system was 97.9% and 84.0% for signal-to-noise ratios (SNRs) of -3 dB and -8 dB, respectively. The proposed system was run in real time, with a real-time factor of 0.378, on a single Kryo 585 Silver core in the RB5 platform. A demo of the proposed system is available at https://youtu.be/3d3Cr-cs9aY.

Estimating the location of a target sound source is an important task in many fields. The blind source separation (BSS) algorithm is closely related to the multiple sound source localization problem. A convolutional beamformer (CBF), one of the BSS algorithms, can successfully estimate the acoustic transfer vector (ATV) for each of the target sources which contains spatial information on the corresponding source. In this paper, we propose an algorithm called SVD-PHAT-ATV that estimates the directions of arrival (DoAs) of multiple sound sources using the ATVs of the CBF. In particular, we extract the DoAs of the target sources from the ATVs by employing the singular value decomposition (SVD) of the steered-response power (SRP) weight matrix. Experimental results on simulation data showed that the proposed method demonstrated better performance and lower computational complexity than the existing BSS-based multiple sound source localization algorithms. In experiments on real-recorded data, the proposed method outperformed the existing first-place algorithm using a robot head microphone array in Task 2 of the LOCATA Challenge.

In this paper, we present a statistical beamforming algorithm as a pre-processing step for robust automatic speech recognition (ASR). By modeling the target speech as a non-stationary Laplacian distribution, a mask-based statistical beamforming algorithm is proposed to exploit both its output and masked input variance for robust estimation of the beamformer. In addition, we also present a method for steering vector estimation (SVE) based on a noise power ratio obtained from the target and noise outputs in independent component analysis (ICA). To update the beamformer in the same ICA framework, we derive ICA with distortionless and null constraints on target speech, which yields beamformed speech at the target output and noises at the other outputs, respectively. The demixing weights for the target output result in a statistical beamformer with the weighted spatial covariance matrix (wSCM) using a weighting function characterized by a source model. To enhance the SVE, the strict null constraints imposed by the Lagrange multiplier methods are relaxed by generalized penalties with weight parameters, while the strict distortionless constraints are maintained. Furthermore, we derive an online algorithm based on an optimization technique of recursive least squares (RLS) for practical applications. Experimental results on various environments using CHiME-4 and LibriCSS datasets demonstrate the effectiveness of the presented algorithm compared to conventional beamforming and blind source extraction (BSE) based on ICA on both batch and online processing.

In this paper, we present a statistical beamforming algorithm as a pre-processing step for robust automatic speech recognition (ASR). By modeling the target speech as a non-stationary Laplacian distribution, a mask-based statistical beamforming algorithm is proposed to exploit both its output and masked input variance for robust estimation of the beamformer. In addition, we also present a method for steering vector estimation (SVE) based on a noise power ratio obtained from the target and noise outputs in independent component analysis (ICA). To update the beamformer in the same ICA framework, we derive ICA with distortionless and null constraints on target speech, which yields beamformed speech at the target output and noises at the other outputs, respectively. The demixing weights for the target output result in a statistical beamformer with the weighted spatial covariance matrix (wSCM) using a weighting function characterized by a source model. To enhance the SVE, the strict null constraints imposed by the Lagrange multiplier methods are relaxed by generalized penalties with weight parameters, while the strict distortionless constraints are maintained. Furthermore, we derive an online algorithm based on an optimization technique of recursive least squares (RLS) for practical applications. Experimental results on various environments using CHiME-4 and LibriCSS datasets demonstrate the effectiveness of the presented algorithm compared to conventional beamforming and blind source extraction (BSE) based on ICA on both batch and online processing.

In speaker verification, ECAPA-TDNN has shown remarkable improvement by utilizing one-dimensional(1D) Res2Net block and squeeze-and-excitation(SE) module, along with multi-layer feature aggregation (MFA). Meanwhile, in vision tasks, ConvNet structures have been modernized by referring to Transformer, resulting in improved performance. In this paper, we present an improved block design for TDNN in speaker verification. Inspired by recent ConvNet structures, we replace the SE-Res2Net block in ECAPA-TDNN with a novel 1D two-step multi-scale ConvNeXt block, which we call TS-ConvNeXt. The TS-ConvNeXt block is constructed using two separated sub-modules: a temporal multi-scale convolution (MSC) and a frame-wise feed-forward network (FFN). This two-step design allows for flexible capturing of inter-frame and intra-frame contexts. Additionally, we introduce global response normalization (GRN) for the FFN modules to enable more selective feature propagation, similar to the SE module in ECAPA-TDNN. Experimental results demonstrate that NeXt-TDNN, with a modernized backbone block, significantly improved performance in speaker verification tasks while reducing parameter size and inference time. We have released our code for future studies.