XLXugang Lu

Xugang Lu

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Xugang Lu has a total of 94 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Domain Adaptation for Speaker Verification Using Optimal Transport With Pseudo Label

    by Wenhao Yang, Jianguo Wei, Wenhuan Lu, Lei Li, Xugang Lu

    Domain gap often degrades the performance of speaker verification (SV) systems when the statistical distributions of training data and real-world test speech are mismatched. Channel variation is a primary factor causing this gap, including bandwidth changes, background noise and encoding, etc. Although various domain adaptation algorithms could be applied to handle this domain gap problem, most algorithms could not take the complex distribution structure in domain alignment with discriminative learning. In this paper, we propose a novel unsupervised domain adaptation method for speaker verification, i.e., Joint Partial Optimal Transport with Pseudo Label (JPOT-PL), to alleviate the domain mismatch problem. Leveraging the geometric-aware distance metric of optimal transport in distribution alignment and speaker consistency in speech distribution, we further design a pseudo label-based discriminative learning where the pseudo label can be regarded as a new type of speaker label derived from the optimal coupling. With the JPOT-PL, we carry out experiments on the SV channel and lingual domain adaptation with VoxCeleb, LibriSpeech, CNCeleb, and AISHELL-2. Experiments show our method reduces EER by up to 30% compared with several state-of-the-art domain adaptation algorithms.

  • A Novel UTI Plane Stability Calibration and Compensation Method Based on Skull Structural Relationships for Articulatory Data Observation

    by Yuxuan Li, Jianguo Wei, Kele Xu, Qiang Fang, Xugang Lu

    The tongue is a crucial articulator, where even minor deformations in shape and position significantly affect speech production. Ultrasound tongue imaging (UTI) has emerged as a crucial method to track complex tongue movements, improving our understanding of how the vocal tract geometry impacts speech production. Superior imaging results also benefit various UTI-based tasks, such as silent speech recognition and speech sound disorder treatment, which rely on imaging plane stability. Sustaining this stability remains challenging, given its impact on UTI consistency and clarity, essential for reproducibility and recognition accuracy. To solve the challenges, we propose a stability calibration and compensation method based on skull structural relationships. This paper disentangles the stability-influencing factors into specific and comprehensible study questions by incorporating the skull structural relationships. It clarifies the mechanism through which the proposed solutions improve stability and image quality. This method is assessed using the improved evaluation framework, which is upgraded by the methods we newly proposed in this study. The evaluation results showcase the effectiveness of our proposed method, showing superior performance over the benchmark. Crucial features, such as the abrupt edge gray change in the UTI images, are provided a preliminary and quantitative representation by integrating our calibration and evaluation methods. The results may further clarify the relationship between the stability, vocal actions, UTI images, energy variations, and improvements. This facilitates more reliable and accurate applications of UTI-based tasks.

  • Progressive Learning Framework With Missing Modality Reconstruction for Multimodal Emotion Recognition

    by Haoqin Sun, Xugang Lu, Jingguang Tian, Jiaming Zhou, Jiabei He, Hui Wang, Xiangyu Kong, Xinhui Hu, Yong Qin

    Multimodal emotion recognition (MMER) is crucial for effective human-computer interaction, yet its performance degrades sharply when modality data are incomplete due to privacy constraints or data loss. Existing approaches often attempt to reconstruct missing information directly from raw, heterogeneous modalities, requiring complex cross-modal mappings that lead to low-quality reconstruction and hence less performance improvement. To address these challenges, we propose a progressive learning framework (ProLF) for missing-modality reconstruction and robust multimodal fusion. ProLF’s core innovation lies in its design of an incremental three-stage strategy, i.e., Alignment, Reconstruction, and Augmentation-which collectively mitigates modality heterogeneity, data incompleteness, and weak fused representations. In the Alignment stage, ProLF applies unsupervised distribution-level contrastive learning to reduce cross-modal discrepancies and establish a unified latent space. In the Reconstruction stage, a normalizing flow model efficiently transforms the aligned latent distributions to recover missing modalities with high accuracy. In the Augmentation stage, a mixture of modality experts performs deep fusion of the restored multimodal information, while supervised point-based contrastive learning enhances emotion-specific features and suppresses irrelevant semantics, substantially improving discriminability. Extensive experiments on multiple benchmark datasets validate the effectiveness of ProLF. On IEMOCAP and MSP-IMPROV, ProLF achieves average performance gains of 1.5%–2%, and on CMU-MOSI, improvements of 1%–3%. These results confirm that ProLF provides robust and consistent benefits under both missing and complete modality conditions.


  • UDA-SD: Unsupervised domain adaptation based on self-distillation for cross-domain audio anti-spoofing

    by Ruiteng Zhang, Xiaohuan Chen, Wenhuan Lu, Xugang Lu, Junhai Xu, Jianguo Wei


  • Open Access

    Combining Deterministic Enhanced Conditions With Dual-Streaming Encoding for Diffusion-Based Speech Enhancement

    by Hao Shi, Xugang Lu, Kazuki Shimada, Tatsuya Kawahara

    Score-based diffusion models for speech enhancement (SE) need to incorporate correct prior knowledge as reliable conditions to generate accurate predictions. However, providing reliable conditions using noisy features is challenging. One solution is to use features enhanced by deterministic methods as conditions. However, the information distortion and loss caused by deterministic methods might affect the diffusion process. In this paper, we first investigate the effects of using different deterministic SE models as conditions for diffusion. We validate two conditions depending on whether the noisy feature was used as part of the condition: one using only the deterministic feature (deterministic-only), and the other using both deterministic and noisy features (deterministic-noisy). Preliminary investigation found that using deterministic enhanced conditions improves DNSMOS and UTMOS on real data, while the choice between using deterministic-only or deterministic-noisy conditions depends on the deterministic models. Based on these findings, we propose the deterministic-diffusion unified model for SE to more effectively utilize both conditions. Moreover, we found that fine-grained deterministic models have greater potential in objective evaluation metrics, while UNet-based deterministic models provide more stable diffusion performance. Therefore, we also propose a deterministic model that combines coarse- and fine-grained processing for the diffusion. Experimental results on CHiME4 show that the proposed models effectively leverage deterministic models to achieve better SE evaluation scores on the DNSMOS and UTMOS for real evaluation sets. In addition, the proposed deterministic model proves to be more stable than other deterministic models when it is used for diffusion.


  • Multi-Sinkhorn Teacher Knowledge Aggregation Framework for Adaptive Audio Anti-Spoofing

    by Ruiteng Zhang, Jianguo Wei, Xugang Lu, Lin Zhang, Di Jin, Wenhuan Lu, Junhai Xu

    Audio anti-spoofing algorithms are widely deployed to defend against spoofing attacks, yet they often fail to detect unseen attacks. Although unsupervised domain adaptation (UDA) offers the potential to address this challenge, existing methods struggle with the large intra-class variability and complex distribution structures in target domains caused by the diversity of speech and attack types. In contrast, optimal transport (OT) leverages the geometric structure of intra-class distributions to measure discrepancies between probability distributions. The effectiveness of OT relies on the discriminability of data within target domains. However, in real-world scenarios involving multiple target domains, these domains often overlap in feature space, leading to the negative transport problem in OT. To overcome these domain mismatches in anti-spoofing, we propose the Multi-Sinkhorn Teacher Knowledge Aggregation (MSTKA) framework. Initially, to avoid interference between target domains during alignment, we use OT to adapt the source model to each target domain independently, thereby reducing negative transport. This adaptation involves constructing an OT cost matrix based on sentence-level representations of cross-domain samples and training an expert model for each target domain. Subsequently, we aggregate the knowledge from these expert models into a unified student model, enabling it to generalize across multiple target domains. Since spoofing cues could be distributed across different temporal scales, we align the student model's representations at multiple time scales with the teacher model's sentence-level representations to enhance the effectiveness of knowledge distillation. Multi-target adaptation experiments on eleven data sets demonstrate that our framework achieves state-of-the-art performance in audio anti-spoofing.


  • SHDA: Sinkhorn Domain Attention for Cross-Domain Audio Anti-Spoofing

    by Ruiteng Zhang, Jianguo Wei, Xugang Lu, Lin Zhang, Di Jin, Junhai Xu, Wenhuan Lu

    Audio anti-spoofing algorithms struggle with fake samples from unseen spoofing techniques, even when trained with diverse data sets or data augmentation strategies. Unsupervised domain adaptation (UDA) algorithms have the potential to mitigate this challenge. Typically, UDA assumes that the source and target domains are distinct distributions with clear boundaries and seeks to align model representations between them. However, in anti-spoofing, various spoofing algorithms could cause the distributions of the generated samples to overlap, resulting in unclear domain boundaries. This hinders UDA algorithms from effectively measuring and aligning domain discrepancies. Moreover, forcibly aligning samples with significant discrepancies could diminish the model’s discriminative capability. To solve this problem, we propose a domain attention algorithm with optimal transport (OT), termed Sinkhorn Domain Attention (SHDA). Unlike traditional attention mechanisms, SHDA identifies the optimal transfer plan by analyzing the global probability differences among cross-domain samples. Specifically, we first extract audio representations from various domains to compute the overall cost matrix between the source and target domains. Next, we employ Sinkhorn’s iteration to calculate the OT coupling matrix, where cross-domain samples with minor differences receive higher transfer weights, while those with substantial differences receive lower weights. Finally, we use the coupling and cost matrices to compute the adaptation loss, effectively transferring the anti-spoofing model from multiple sources to the target domain. We conducted eight cross-domain experiments using eleven well-known anti-spoofing corpora. The results indicate that our label-free SHDA surpassed the state-of-the-art model by 40%.


  • Open Access

    Cross-modal Knowledge Transfer Learning as Graph Matching Based on Optimal Transport for ASR

    by Xugang Lu, Peng Shen, Yu Tsao, Hisashi Kawai

    Transferring linguistic knowledge from a pretrained language model (PLM) to acoustic feature learning has proven effective in enhancing end-to-end automatic speech recognition (E2E-ASR). However, aligning representations between linguistic and acoustic modalities remains a challenge due to inherent modality gaps. Optimal transport (OT) has shown promise in mitigating these gaps by minimizing the Wasserstein distance (WD) between linguistic and acoustic feature distributions. However, previous OT-based methods overlook structural relationships, treating feature vectors as unordered sets. To address this, we propose Graph Matching Optimal Transport (GM-OT), which models linguistic and acoustic sequences as structured graphs. Nodes represent feature embeddings, while edges capture temporal and sequential relationships. GM-OT minimizes both WD (between nodes) and Gromov-Wasserstein distance (GWD) (between edges), leading to a fused Gromov-Wasserstein distance (FGWD) formulation. This enables structured alignment and more efficient knowledge transfer compared to existing OT-based approaches. Theoretical analysis further shows that prior OT-based methods in linguistic knowledge transfer can be viewed as a special case within our GM-OT framework. We evaluate GM-OT on Mandarin ASR using a CTC-based E2E-ASR system with a PLM for knowledge transfer. Experimental results demonstrate significant performance gains over state-of-the-art models, validating the effectiveness of our approach.


  • An adaptation framework with unified embedding reconstruction for cross-corpus speech emotion recognition

    by Ruiteng Zhang, Jianguo Wei, Xugang Lu, Yongwei Li, Wenhuan Lu, Lin Zhang, Junhai Xu