Qing Wang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Qing Wang has a total of 59 co-authors. Here are the top co-authors by number of shared publications:
- Jun Du
University of Science and Technology of China, Hefei, Anhui, China
16 shared publications - Hang Chen
University of Science and Technology of China, Hefei, Anhui, China
8 shared publications - Chin-Hui Lee
School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, USA
7 shared publications - Genshun Wan
National Engineering Research Center of Speech and Language Information Processing, University of Science and Technology of China, Hefei, China
3 shared publications Jia Pan
3 shared publications- Sabato Marco Siniscalchi
University of Palermo, Palermo, Italy
3 shared publications
Latest Publications
- 15 May 2026
- Published by Elsevier in Pattern Recognition
- Vol. 180
Reinforcement learning-powered co-optimization: Bridging critic model and multimodal LLM reasoning abilities
- 15 May 2026
- Published by Elsevier in Pattern Recognition
- Vol. 180
Reinforcement learning-powered co-optimization: Bridging critic model and multimodal LLM reasoning abilities
- 25 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech and Language Processing
- Vol. 34, 1915-1928
HDA-SELD: Hierarchical Cross-Modal Distillation With Multi-Level Data Augmentation for Low-Resource Audio-Visual Sound Event Localization and Detection
This work presents HDA-SELD, a unified hierarchical distillation and augmentation framework for audio-visual (AV) sound event localization and detection (SELD) designed to address the challenge of data scarcity. The proposed framework integrates hierarchical cross-modal distillation (HCMD) to transfer knowledge from a robust audio-only SELD teacher to an AV student through both output responses and intermediate hidden representations. To enhance learning, we introduce a multi-level data augmentation strategy that mixes features randomly selected from multiple network layers and associated loss functions tailored to the SELD task. By employing loss interpolation instead of direct label manipulation, the strategy ensures spatial consistency during the augmentation process. Extensive experiments on the DCASE 2023 and 2024 Challenge SELD datasets show that the proposed method significantly improves AV SELD performance, yielding relative gains of 21%-38% in the overall metric over the baselines. Notably, our proposed HDA-SELD achieves results comparable to or better than teacher models trained on much larger datasets, surpassing state-of-the-art methods on both DCASE 2023 and 2024 Challenge SELD tasks.
- 25 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech and Language Processing
- Vol. 34, 1915-1928
HDA-SELD: Hierarchical Cross-Modal Distillation With Multi-Level Data Augmentation for Low-Resource Audio-Visual Sound Event Localization and Detection
This work presents HDA-SELD, a unified hierarchical distillation and augmentation framework for audio-visual (AV) sound event localization and detection (SELD) designed to address the challenge of data scarcity. The proposed framework integrates hierarchical cross-modal distillation (HCMD) to transfer knowledge from a robust audio-only SELD teacher to an AV student through both output responses and intermediate hidden representations. To enhance learning, we introduce a multi-level data augmentation strategy that mixes features randomly selected from multiple network layers and associated loss functions tailored to the SELD task. By employing loss interpolation instead of direct label manipulation, the strategy ensures spatial consistency during the augmentation process. Extensive experiments on the DCASE 2023 and 2024 Challenge SELD datasets show that the proposed method significantly improves AV SELD performance, yielding relative gains of 21%-38% in the overall metric over the baselines. Notably, our proposed HDA-SELD achieves results comparable to or better than teacher models trained on much larger datasets, surpassing state-of-the-art methods on both DCASE 2023 and 2024 Challenge SELD tasks.
- 27 January 2026
- Published by Elsevier in Neurocomputing
- Vol. 673
See then tell: Enhancing key information extraction with vision grounding
- 27 January 2026
- Published by Elsevier in Neurocomputing
- Vol. 673
See then tell: Enhancing key information extraction with vision grounding
- Open Access
- 26 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 11-20
MMC: Iterative Refinement of VLM Reasoning via MCTS-based Multimodal Critique
- 26 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 11-20
Open AccessMMC: Iterative Refinement of VLM Reasoning via MCTS-based Multimodal Critique
- 7 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 27 (15209210) , 6489-6502
Video Segmentation and Tokenization for Model-Based Video Scene Classification
In this paper, we propose a novel approach for segmenting and tokenizing a video scene recording into a sequence of cascade units, known as visual segment units and modeled with visual segment models (VSMs) for video scene classification (VSC). Specifically, the proposed VSM framework takes deep visual features extracted from pre-trained encoders as inputs and models the temporal interactions between segment units by hidden Markov models. Next, we use unit co-occurrence statistics to introduce relationships between VSM units within a video scene recording. Furthermore, the VSM approach is extended to an acoustic-visual variant, subsequently integrating itself into a deep learning-based multi-modal scene classification system. This combination serves to further exploit the complementary nature of audio and video data. By incorporating a set of visual segment units into modeling a video scene class, it captures both inter-class similarity and intra-class diversity, facilitating improved scene classification, especially within categories prone to confusion. Extensive experimental results on a benchmark published by the DCASE (Detection and Classification of Acoustic Scenes and Events) 2021 Challenge show that the proposed framework can effectively handle the confusion issue among similar video scenes. In addition, our multi-modal integration system achieves state-of-the-art performance in the audio-visual scene classification task in the DCASE 2021 Challenge, thereby demonstrating the effectiveness of our proposed approach.
- 7 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 27 (15209210) , 6489-6502
Video Segmentation and Tokenization for Model-Based Video Scene Classification
In this paper, we propose a novel approach for segmenting and tokenizing a video scene recording into a sequence of cascade units, known as visual segment units and modeled with visual segment models (VSMs) for video scene classification (VSC). Specifically, the proposed VSM framework takes deep visual features extracted from pre-trained encoders as inputs and models the temporal interactions between segment units by hidden Markov models. Next, we use unit co-occurrence statistics to introduce relationships between VSM units within a video scene recording. Furthermore, the VSM approach is extended to an acoustic-visual variant, subsequently integrating itself into a deep learning-based multi-modal scene classification system. This combination serves to further exploit the complementary nature of audio and video data. By incorporating a set of visual segment units into modeling a video scene class, it captures both inter-class similarity and intra-class diversity, facilitating improved scene classification, especially within categories prone to confusion. Extensive experimental results on a benchmark published by the DCASE (Detection and Classification of Acoustic Scenes and Events) 2021 Challenge show that the proposed framework can effectively handle the confusion issue among similar video scenes. In addition, our multi-modal integration system achieves state-of-the-art performance in the audio-visual scene classification task in the DCASE 2021 Challenge, thereby demonstrating the effectiveness of our proposed approach.
- 24 April 2025
- Published by Elsevier in Information Fusion
- Vol. 122
Cross-attention among spectrum, waveform and SSL representations with bidirectional knowledge distillation for speech enhancement
- 24 April 2025
- Published by Elsevier in Information Fusion
- Vol. 122
Cross-attention among spectrum, waveform and SSL representations with bidirectional knowledge distillation for speech enhancement
- 10 April 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Signal Processing
- Vol. 19 (4) , 671-684
HPCNet: Hybrid Pixel and Contour Network for Audio-Visual Speech Enhancement With Low-Quality Video
To advance audio-visual speech enhancement (AVSE) research in low-quality video settings, we introduce the multimodal information-based speech processing-low quality video (MISP-LQV) benchmark, which includes a 120-hour real-world Mandarin audio-visual dataset, two video degradation simulation methods, and benchmark results from several well-known AVSE models. We also propose a novel hybrid pixel and contour network (HPCNet), incorporating a lip reconstruction and distillation (LRD) module and a contour graph convolution (CGConv) layer. Specifically, the LRD module reconstructs high-quality lip frames from low-quality audio-visual data, utilizing knowledge distillation from a teacher model trained on high-quality data. The CGConv layer employs spatio-temporal and semantic-contextual graphs to capture complex relationships among lip landmark points. Extensive experiments on the MISP-LQV benchmark reveal the performance degradation caused by low-quality video across various AVSE models. Notably, including real/simulated low-quality videos in AVSE training enhances its robustness to low-quality videos but degrades the performance of high-quality videos.The proposed HPCNet demonstrates strong robustness against video quality degradation, which can be attributed to (1) the reconstructed lip frames closely aligning with high-quality frames and (2) the contour features exhibiting consistency across different video quality levels. The generalizability of HPCNet also has been validated through experiments on the 2nd COG-MHEAR AVSE Challenge dataset. 11Dataset and source code will be available at https://github.com/coal-boss/HPCNet.
- 10 April 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Signal Processing
- Vol. 19 (4) , 671-684
HPCNet: Hybrid Pixel and Contour Network for Audio-Visual Speech Enhancement With Low-Quality Video
To advance audio-visual speech enhancement (AVSE) research in low-quality video settings, we introduce the multimodal information-based speech processing-low quality video (MISP-LQV) benchmark, which includes a 120-hour real-world Mandarin audio-visual dataset, two video degradation simulation methods, and benchmark results from several well-known AVSE models. We also propose a novel hybrid pixel and contour network (HPCNet), incorporating a lip reconstruction and distillation (LRD) module and a contour graph convolution (CGConv) layer. Specifically, the LRD module reconstructs high-quality lip frames from low-quality audio-visual data, utilizing knowledge distillation from a teacher model trained on high-quality data. The CGConv layer employs spatio-temporal and semantic-contextual graphs to capture complex relationships among lip landmark points. Extensive experiments on the MISP-LQV benchmark reveal the performance degradation caused by low-quality video across various AVSE models. Notably, including real/simulated low-quality videos in AVSE training enhances its robustness to low-quality videos but degrades the performance of high-quality videos.The proposed HPCNet demonstrates strong robustness against video quality degradation, which can be attributed to (1) the reconstructed lip frames closely aligning with high-quality frames and (2) the contour features exhibiting consistency across different video quality levels. The generalizability of HPCNet also has been validated through experiments on the 2nd COG-MHEAR AVSE Challenge dataset. 11Dataset and source code will be available at https://github.com/coal-boss/HPCNet.
- 4 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 32 (23299290) , 3025-3038
A Variance-Preserving Interpolation Approach for Diffusion Models With Applications to Single Channel Speech Enhancement and Recognition
In this paper, we propose a variance-preserving interpolation framework to improve diffusion models for single-channel speech enhancement (SE) and automatic speech recognition (ASR). This new variance-preserving interpolation diffusion model (VPIDM) approach requires only 25 iterative steps and obviates the need for a corrector, an essential element in the existing variance-exploding interpolation diffusion model (VEIDM). Two notable distinctions between VPIDM and VEIDM are the scaling function of the mean of state variables and the constraint imposed on the variance relative to the mean's scale. We conduct a systematic exploration of the theoretical mechanism underlying VPIDM, and develop insights regarding VPIDM's applications in SE and ASR using VPIDM as a frontend. Our proposed approach, evaluated on two distinct data sets, demonstrates VPIDM's superior performances over conventional discriminative SE algorithms. Furthermore, we assess the performance of the proposed model under varying signal-to-noise ratio (SNR) levels. The investigation reveals VPIDM's improved robustness in target noise elimination when compared to VEIDM. Furthermore, utilizing the mid-outputs of both VPIDM and VEIDM results in enhanced ASR accuracies, thereby highlighting the practical efficacy of our proposed approach. Code and audio examples are available online https://github.com/zelokuo/VPIDM .
- 4 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 32 (23299290) , 3025-3038
A Variance-Preserving Interpolation Approach for Diffusion Models With Applications to Single Channel Speech Enhancement and Recognition
In this paper, we propose a variance-preserving interpolation framework to improve diffusion models for single-channel speech enhancement (SE) and automatic speech recognition (ASR). This new variance-preserving interpolation diffusion model (VPIDM) approach requires only 25 iterative steps and obviates the need for a corrector, an essential element in the existing variance-exploding interpolation diffusion model (VEIDM). Two notable distinctions between VPIDM and VEIDM are the scaling function of the mean of state variables and the constraint imposed on the variance relative to the mean's scale. We conduct a systematic exploration of the theoretical mechanism underlying VPIDM, and develop insights regarding VPIDM's applications in SE and ASR using VPIDM as a frontend. Our proposed approach, evaluated on two distinct data sets, demonstrates VPIDM's superior performances over conventional discriminative SE algorithms. Furthermore, we assess the performance of the proposed model under varying signal-to-noise ratio (SNR) levels. The investigation reveals VPIDM's improved robustness in target noise elimination when compared to VEIDM. Furthermore, utilizing the mid-outputs of both VPIDM and VEIDM results in enhanced ASR accuracies, thereby highlighting the practical efficacy of our proposed approach. Code and audio examples are available online https://github.com/zelokuo/VPIDM .