Haoqin Sun
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Haoqin Sun has a total of 24 co-authors. Here are the top co-authors by number of shared publications:
- Hui Wang
College of Computer Science, Nankai University, Tianjin, China
3 shared publications - Jiaming Zhou
College of Computer Science, Nankai University, Tianjin, China
3 shared publications - Yong Qin
College of Computer Science, Nankai University, Tianjin, China
3 shared publications - Shiwan Zhao
College of Computer Science, Nankai University, Tianjin, China
2 shared publications - Wenbo Guan
School of Information Science and Technology, Qingdao University of Science and Technology, Qingdao, China
2 shared publications - Yang Liu
Qingdao University of Science and Technology
2 shared publications
Latest Publications
- 9 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech and Language Processing
- Vol. 34, 1538-1551
KNN-CTC$+$: Retrieval-Augmented Decoding for Robust CTC-Based ASR
Retrieval-augmented methods complement parametric models with non-parametric memory and have shown strong adaptability in natural language processing. Extending this paradigm to automatic speech recognition (ASR) is appealing but challenging, particularly at the frame level where precise alignments are difficult to obtain and large-scale audio-text datastores impose heavy storage and retrieval costs. We present$k$ NN-CTC$+$ , a retrieval-augmented decoding framework for CTC-based ASR that addresses these challenges. To construct fine-grained datastores without external supervision,$k$ NN-CTC$+$ leverages CTC encoder embeddings as keys and frame-level pseudo labels as values. To reduce redundancy, we propose a blank-aware pruning strategy that removes frames during both datastore construction and query-time retrieval, improving efficiency while retaining informative evidence. Beyond alignment and scale issues, domain mismatch remains a major obstacle.$k$ NN-CTC$+$ introduces a lightweight unsupervised domain adaptation (UDA) pipeline: confidence-based filtering discards unreliable pseudo labels when building target-specific datastores, and a dynamic interpolation strategy adaptively balances model predictions and$k$ NN retrievals based on neighbor confidence. These designs enable robust test-time adaptation without labeled data or parameter updates. Experiments on mandarin, Chinese dialect, child and elderly speech datasets demonstrate consistent accuracy gains and improved robustness under domain shifts.$k$ NN-CTC$+$ provides a scalable, effective solution for enhancing CTC-based ASR. Code is available at https://github.com/NKU-HLT/KNN-CTC.- 9 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech and Language Processing
- Vol. 34, 1538-1551
KNN-CTC$+$: Retrieval-Augmented Decoding for Robust CTC-Based ASR
Retrieval-augmented methods complement parametric models with non-parametric memory and have shown strong adaptability in natural language processing. Extending this paradigm to automatic speech recognition (ASR) is appealing but challenging, particularly at the frame level where precise alignments are difficult to obtain and large-scale audio-text datastores impose heavy storage and retrieval costs. We present$k$ NN-CTC$+$ , a retrieval-augmented decoding framework for CTC-based ASR that addresses these challenges. To construct fine-grained datastores without external supervision,$k$ NN-CTC$+$ leverages CTC encoder embeddings as keys and frame-level pseudo labels as values. To reduce redundancy, we propose a blank-aware pruning strategy that removes frames during both datastore construction and query-time retrieval, improving efficiency while retaining informative evidence. Beyond alignment and scale issues, domain mismatch remains a major obstacle.$k$ NN-CTC$+$ introduces a lightweight unsupervised domain adaptation (UDA) pipeline: confidence-based filtering discards unreliable pseudo labels when building target-specific datastores, and a dynamic interpolation strategy adaptively balances model predictions and$k$ NN retrievals based on neighbor confidence. These designs enable robust test-time adaptation without labeled data or parameter updates. Experiments on mandarin, Chinese dialect, child and elderly speech datasets demonstrate consistent accuracy gains and improved robustness under domain shifts.$k$ NN-CTC$+$ provides a scalable, effective solution for enhancing CTC-based ASR. Code is available at https://github.com/NKU-HLT/KNN-CTC.- 5 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech and Language Processing
- Vol. 34, 29-43
Progressive Learning Framework With Missing Modality Reconstruction for Multimodal Emotion Recognition
Multimodal emotion recognition (MMER) is crucial for effective human-computer interaction, yet its performance degrades sharply when modality data are incomplete due to privacy constraints or data loss. Existing approaches often attempt to reconstruct missing information directly from raw, heterogeneous modalities, requiring complex cross-modal mappings that lead to low-quality reconstruction and hence less performance improvement. To address these challenges, we propose a progressive learning framework (ProLF) for missing-modality reconstruction and robust multimodal fusion. ProLF’s core innovation lies in its design of an incremental three-stage strategy, i.e., Alignment, Reconstruction, and Augmentation-which collectively mitigates modality heterogeneity, data incompleteness, and weak fused representations. In the Alignment stage, ProLF applies unsupervised distribution-level contrastive learning to reduce cross-modal discrepancies and establish a unified latent space. In the Reconstruction stage, a normalizing flow model efficiently transforms the aligned latent distributions to recover missing modalities with high accuracy. In the Augmentation stage, a mixture of modality experts performs deep fusion of the restored multimodal information, while supervised point-based contrastive learning enhances emotion-specific features and suppresses irrelevant semantics, substantially improving discriminability. Extensive experiments on multiple benchmark datasets validate the effectiveness of ProLF. On IEMOCAP and MSP-IMPROV, ProLF achieves average performance gains of 1.5%–2%, and on CMU-MOSI, improvements of 1%–3%. These results confirm that ProLF provides robust and consistent benefits under both missing and complete modality conditions.
- 5 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech and Language Processing
- Vol. 34, 29-43
Progressive Learning Framework With Missing Modality Reconstruction for Multimodal Emotion Recognition
Multimodal emotion recognition (MMER) is crucial for effective human-computer interaction, yet its performance degrades sharply when modality data are incomplete due to privacy constraints or data loss. Existing approaches often attempt to reconstruct missing information directly from raw, heterogeneous modalities, requiring complex cross-modal mappings that lead to low-quality reconstruction and hence less performance improvement. To address these challenges, we propose a progressive learning framework (ProLF) for missing-modality reconstruction and robust multimodal fusion. ProLF’s core innovation lies in its design of an incremental three-stage strategy, i.e., Alignment, Reconstruction, and Augmentation-which collectively mitigates modality heterogeneity, data incompleteness, and weak fused representations. In the Alignment stage, ProLF applies unsupervised distribution-level contrastive learning to reduce cross-modal discrepancies and establish a unified latent space. In the Reconstruction stage, a normalizing flow model efficiently transforms the aligned latent distributions to recover missing modalities with high accuracy. In the Augmentation stage, a mixture of modality experts performs deep fusion of the restored multimodal information, while supervised point-based contrastive learning enhances emotion-specific features and suppresses irrelevant semantics, substantially improving discriminability. Extensive experiments on multiple benchmark datasets validate the effectiveness of ProLF. On IEMOCAP and MSP-IMPROV, ProLF achieves average performance gains of 1.5%–2%, and on CMU-MOSI, improvements of 1%–3%. These results confirm that ProLF provides robust and consistent benefits under both missing and complete modality conditions.
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 10229-10238
FELLE: Autoregressive Speech Synthesis with Token-Wise Coarse-to-Fine Flow Matching
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 10229-10238
Open AccessFELLE: Autoregressive Speech Synthesis with Token-Wise Coarse-to-Fine Flow Matching
- 1 June 2023
- Published by Springer Nature in Machine Intelligence Research
- Vol. 20 (4) , 595-604
Speech Emotion Recognition Using Cascaded Attention Network with Joint Loss for Discrimination of Confusions
- 1 June 2023
- Published by Springer Nature in Machine Intelligence Research
- Vol. 20 (4) , 595-604
Speech Emotion Recognition Using Cascaded Attention Network with Joint Loss for Discrimination of Confusions
- 15 February 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 31 (23299290) , 1063-1074
A Discriminative Feature Representation Method Based on Cascaded Attention Network With Adversarial Strategy for Speech Emotion Recognition
Currently, speech emotion recognition models still could not show satisfactory performance due to the complexity of emotions. In most of the previous studies, there is a common problem that some of the particular emotions are severely misclassified. In this paper, we propose a novel framework integrating cascaded attention network and adversarial joint loss strategy for speech emotion recognition, aiming at discriminating the confusions by emphasizing more on the emotions which are difficult to be correctly classified. First, we extract log-Mels, deltas and delta-deltas of log-Mels as 3D features to effectively reduce the interference of external factors. Next, we introduce a cascaded attention network to extract effective emotional features, where spatiotemporal attention selectively locates the targeted emotional regions from the input features. In these targeted regions, the self attention with head fusion captures the long-distance dependence of temporal features. Finally, an adversarial joint loss strategy is proposed to distinguish the emotional embeddings with high similarity by the generated hard triplets in an adversarial fashion. To evaluate our proposed method, experiments are performed with the IEMOCAP, CASIA, and EMODB corpora. The experimental results demonstrate that our proposed method significantly outperforms the state-of-the-art approaches on all datasets.
- 15 February 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 31 (23299290) , 1063-1074
A Discriminative Feature Representation Method Based on Cascaded Attention Network With Adversarial Strategy for Speech Emotion Recognition
Currently, speech emotion recognition models still could not show satisfactory performance due to the complexity of emotions. In most of the previous studies, there is a common problem that some of the particular emotions are severely misclassified. In this paper, we propose a novel framework integrating cascaded attention network and adversarial joint loss strategy for speech emotion recognition, aiming at discriminating the confusions by emphasizing more on the emotions which are difficult to be correctly classified. First, we extract log-Mels, deltas and delta-deltas of log-Mels as 3D features to effectively reduce the interference of external factors. Next, we introduce a cascaded attention network to extract effective emotional features, where spatiotemporal attention selectively locates the targeted emotional regions from the input features. In these targeted regions, the self attention with head fusion captures the long-distance dependence of temporal features. Finally, an adversarial joint loss strategy is proposed to distinguish the emotional embeddings with high similarity by the generated hard triplets in an adversarial fashion. To evaluate our proposed method, experiments are performed with the IEMOCAP, CASIA, and EMODB corpora. The experimental results demonstrate that our proposed method significantly outperforms the state-of-the-art approaches on all datasets.