UniArray: Unified Spectral-Spatial Modeling for Array-Geometry-Agnostic Speech Separation
- 21 April 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Signal Processing Letters
- Vol. 32 (10709908) , 2164-2168
Abstract
Array-geometry-agnostic speech separation (AGA-SS) aims to develop an effective separation method regardless of the microphone array geometry. Conventional methods rely on permutation-free operations, such as summation or attention mechanisms, to capture spatial information. However, these approaches often incur high computational costs or disrupt the effective use of spatial information during intra- and inter-channel interactions, leading to suboptimal performance. To address these issues, we propose UniArray, a novel approach that abandons the conventional interleaving manner. UniArray consists of three key components: a virtual microphone estimation (VME) module, a feature extraction and fusion module, and a hierarchical dual-path separator. The VME ensures robust performance across arrays with varying channel numbers. The feature extraction and fusion module leverages a spectral feature extraction module and a spatial dictionary learning (SDL) module to extract and fuse frequency-bin-level features, allowing the separator to focus on using the fused features. The hierarchical dual-path separator models feature dependencies along the time and frequency axes while maintaining computational efficiency. Experimental results show that UniArray outperforms state-of-the-art methods in SI-SDRi, WB-PESQ, NB-PESQ, and STOI across both seen and unseen array geometries.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (61971186)
This publication has 39 references indexed in Scilit:
- VarArray: Array-Geometry-Agnostic Continuous Speech Separation2022
- Neural Network-Based Virtual Microphone Estimator2021
- Attention Is All You Need In Speech Separation2021
- End-to-end Microphone Permutation and Number Invariant Multi-channel Speech Separation2020
- Overlap-Aware Diarization: Resegmentation Using Neural End-to-End Overlapped Speech Detection2020
- Enhancing End-to-End Multi-Channel Speech Separation Via Spatial Feature Learning2020
- Dual-Path RNN: Efficient Long Sequence Modeling for Time-Domain Single-Channel Speech Separation2020
- SDR – Half-baked or Well Done?2019
- Deep clustering: Discriminative embeddings for segmentation and separation2016
- Perceptual evaluation of speech quality (PESQ)-a new method for speech quality assessment of telephone networks and codecs2002