Xiaolin Jiao

At present, a prevalent approach to speaker diarization is clustering based on speaker embeddings. However, this method encounters two primary issues. Firstly, it cannot directly minimize the diarization error during the training process; secondly, the majority of clustering-based methods struggle to handle speaker overlap in audio. A viable approach for addressing these issues involves adopting end-to-end speaker diarization (EEND). Nevertheless, training this EEND system generally requires lengthy audio inputs, which must be downsampled to allow efficient model processing. In this study, we develop a novel downsampling layer using blueprint separable convolution (BSConv) instead of depthwise separable convolution (DSC) as the foundational convolutional unit, which effectively preserves information from the original audio. Furthermore, we incorporate multi-scale feature aggregation (MFA) into the encoder structure to combine the features extracted by each conformer block to the output layer, consequently enhancing the expressiveness of the model’s feature extraction. Lastly, we employ the conformer as the backbone network to incorporate the proposed enhancements, resulting in an EEND system named BSAC-EEND. We assess our suggested methodology on both simulated and real datasets. The experiment indicates that our proposed EEND system reduces diarization error rate (DER) by an average of 17.3% for two-speaker datasets and 12.8% for three-speaker datasets compared to the baseline.

In this paper, we propose MixLoss, a simple and effective data augmentation method for sequence generation tasks like machine translation, speech recognition, etc. MixLoss works by mixing two inputs and two losses with two corresponding outputs using the same weights, which can also be applied in the hidden space. Concretely, when the loss takes the certain form that we define, we prove that MixLoss is theoretically equivalent to Mixup and thus has an adequate theoretical foundation. We corroborate our theoretical analysis with experiments that support our conclusions. Experimental results on the English-Germany dataset reveal that MixLoss achieves the best performance compared with other data augmentation methods, and can integrate well with other technologies. Specifically, the BLEU can be improved from 27.10 to 28.41 on the MT task, and the WER can be decreased from 18.9% to 15.1% on the ASR task. Moreover, MixLoss has stronger robustness to resist noisy data.