Jianqing Gao
Jianqing Gao has a total of 32 co-authors. Here are the top co-authors by number of shared publications:
- Jun Du
University of Science and Technology of China, Hefei, Anhui, China
7 shared publications - Feng Ma
FLYTEK Research, Hefei, China
2 shared publications - Jia Pan
FLYTEK Research, Hefei, China
2 shared publications - Jianshu Zhang
FLYTEK Research, Hefei, China
2 shared publications - Jiefeng Ma
University of Science and Technology of China, Hefei, China
2 shared publications - Pengfei Hu
University of Science and Technology of China, Hefei, China
2 shared publications
Latest Publications
- 25 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech and Language Processing
- Vol. 34, 1915-1928
HDA-SELD: Hierarchical Cross-Modal Distillation With Multi-Level Data Augmentation for Low-Resource Audio-Visual Sound Event Localization and Detection
This work presents HDA-SELD, a unified hierarchical distillation and augmentation framework for audio-visual (AV) sound event localization and detection (SELD) designed to address the challenge of data scarcity. The proposed framework integrates hierarchical cross-modal distillation (HCMD) to transfer knowledge from a robust audio-only SELD teacher to an AV student through both output responses and intermediate hidden representations. To enhance learning, we introduce a multi-level data augmentation strategy that mixes features randomly selected from multiple network layers and associated loss functions tailored to the SELD task. By employing loss interpolation instead of direct label manipulation, the strategy ensures spatial consistency during the augmentation process. Extensive experiments on the DCASE 2023 and 2024 Challenge SELD datasets show that the proposed method significantly improves AV SELD performance, yielding relative gains of 21%-38% in the overall metric over the baselines. Notably, our proposed HDA-SELD achieves results comparable to or better than teacher models trained on much larger datasets, surpassing state-of-the-art methods on both DCASE 2023 and 2024 Challenge SELD tasks.
- 25 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech and Language Processing
- Vol. 34, 1915-1928
HDA-SELD: Hierarchical Cross-Modal Distillation With Multi-Level Data Augmentation for Low-Resource Audio-Visual Sound Event Localization and Detection
This work presents HDA-SELD, a unified hierarchical distillation and augmentation framework for audio-visual (AV) sound event localization and detection (SELD) designed to address the challenge of data scarcity. The proposed framework integrates hierarchical cross-modal distillation (HCMD) to transfer knowledge from a robust audio-only SELD teacher to an AV student through both output responses and intermediate hidden representations. To enhance learning, we introduce a multi-level data augmentation strategy that mixes features randomly selected from multiple network layers and associated loss functions tailored to the SELD task. By employing loss interpolation instead of direct label manipulation, the strategy ensures spatial consistency during the augmentation process. Extensive experiments on the DCASE 2023 and 2024 Challenge SELD datasets show that the proposed method significantly improves AV SELD performance, yielding relative gains of 21%-38% in the overall metric over the baselines. Notably, our proposed HDA-SELD achieves results comparable to or better than teacher models trained on much larger datasets, surpassing state-of-the-art methods on both DCASE 2023 and 2024 Challenge SELD tasks.
- 19 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech and Language Processing
- Vol. 34, 1966-1979
Closing the ELBO Gap in Diffusion Models for Speech Enhancement and Dereverberation
We examine the Evidence Lower Bound (ELBO) within diffusion models (DMs) applied to speech enhancement (SE) and dereverberation (SD). We focus in particular on the interplay between the ELBO and Gaussian noise schedule (GNS), and the choice of practical loss functions. We hypothesize that the suboptimal performance of DM-based SE and SD can arise from the absence of a well-calibrated GNS. We therefore refine the noise schedule design by controlling the minimum and maximum noise variances. Additionally, we introduce the Importance of Condition as a novel metric that quantitatively assesses the influence of noise variance on the model behavior during reverse diffusion processes. Our analysis reveals that changing the GNS configuration substantially affects the reliance of the model on the input condition, thereby impacting the overall performance. Furthermore, we demonstrate that conventional loss functions used in DMs inherently impose a performance ceiling that prevents convergence to the theoretical optimum of the ELBO, resulting in suboptimal SE and SD outcomes. We propose a two-stage training framework to alleviate this limit. First, a score-based DM uses an optimized GNS to perform initial enhancement. Second, a dedicated refinement model is trained to further improve the ELBO and enhance speech quality. Our comprehensive experimental validation demonstrates the effectiveness of the proposed framework on both SE and SD tasks.
- 19 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech and Language Processing
- Vol. 34, 1966-1979
Closing the ELBO Gap in Diffusion Models for Speech Enhancement and Dereverberation
We examine the Evidence Lower Bound (ELBO) within diffusion models (DMs) applied to speech enhancement (SE) and dereverberation (SD). We focus in particular on the interplay between the ELBO and Gaussian noise schedule (GNS), and the choice of practical loss functions. We hypothesize that the suboptimal performance of DM-based SE and SD can arise from the absence of a well-calibrated GNS. We therefore refine the noise schedule design by controlling the minimum and maximum noise variances. Additionally, we introduce the Importance of Condition as a novel metric that quantitatively assesses the influence of noise variance on the model behavior during reverse diffusion processes. Our analysis reveals that changing the GNS configuration substantially affects the reliance of the model on the input condition, thereby impacting the overall performance. Furthermore, we demonstrate that conventional loss functions used in DMs inherently impose a performance ceiling that prevents convergence to the theoretical optimum of the ELBO, resulting in suboptimal SE and SD outcomes. We propose a two-stage training framework to alleviate this limit. First, a score-based DM uses an optimized GNS to perform initial enhancement. Second, a dedicated refinement model is trained to further improve the ELBO and enhance speech quality. Our comprehensive experimental validation demonstrates the effectiveness of the proposed framework on both SE and SD tasks.
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Signal Processing Letters
- Vol. 33, 4371-4375
Emotion-Centric Speech Pretraining via Knowledge-Augmented Masked Autoencoder With Pronunciation Guidance
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Signal Processing Letters
- Vol. 33, 4371-4375
Emotion-Centric Speech Pretraining via Knowledge-Augmented Masked Autoencoder With Pronunciation Guidance
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 13266-13272
AudioAtlas: A Comprehensive and Balanced Benchmark Towards Movie-Oriented Text-to-Audio Generation
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 13266-13272
Open AccessAudioAtlas: A Comprehensive and Balanced Benchmark Towards Movie-Oriented Text-to-Audio Generation
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 5394-5403
Enhancing the Geometric Problem-Solving Ability of Multimodal LLMs via Symbolic-Neural Integration
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 5394-5403
Open AccessEnhancing the Geometric Problem-Solving Ability of Multimodal LLMs via Symbolic-Neural Integration
- Open Access
- 26 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 11-20
MMC: Iterative Refinement of VLM Reasoning via MCTS-based Multimodal Critique
- 26 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 11-20
Open AccessMMC: Iterative Refinement of VLM Reasoning via MCTS-based Multimodal Critique
- 1 January 2022
- Published by Inderscience Publishers in International Journal of Information and Computer Security
- Vol. 18 (1/2) , 204
An intelligent stage light-based actor identification and positioning system
- 1 January 2022
- Published by Inderscience Publishers in International Journal of Information and Computer Security
- Vol. 18 (1/2) , 204
An intelligent stage light-based actor identification and positioning system
- 30 August 2019
- Published by VM Media Group sp. z o.o in Neurologia i Neurochirurgia Polska
- Vol. 53 (4) , 265-270
Clinical value of 99Tcm-MIBI gated myocardial perfusion imaging in evaluating sarcoglycanopathy
Aim. The purpose of this study was to analyse the diagnostic value of gated myocardial perfusion imaging (G-MPI) in the evaluation of myocardial injury in sarcoglycanopathy. Materials and methods. Twenty-eight patients diagnosed with sarcoglycanopathy were evaluated using 99m- -methoxyisobutylisonitrile(99Tcm-MIBI) G-MPI. The data was processed into tomographic images, and the left ventricular function was analysed using quantitative gated SPECT (QGS) to assess the degree of impairment in myocardial and cardiac function. Results. The images of 23 of the patients (82.1%) were positive. Two hundred and twenty-nine sub-segments with abnormal lesions were detected out of 391 cardiac sub-segments of these 23 positive cases. According to the segmental abnormalities, the cases were divided into two cases (8.7%) with single abnormal wall segment, six cases (26.1%) with two abnormal wall segments, and 15 cases (65.2%) with three or more abnormal wall segments or scattered lesions. Conclusions. 99Tcm-MIBI G-MPI can objectively show impaired myocardium in patients with sarcoglycanopathy. Therefore, this method is helpful for early diagnosis and follow-up of myocardial damage. Aim. The purpose of this study was to analyse the diagnostic value of gated myocardial perfusion imaging (G-MPI) in the evaluation of myocardial injury in sarcoglycanopathy. Materials and methods. Twenty-eight patients diagnosed with sarcoglycanopathy were evaluated using 99m- -methoxyisobutylisonitrile(99Tcm-MIBI) G-MPI. The data was processed into tomographic images, and the left ventricular function was analysed using quantitative gated SPECT (QGS) to assess the degree of impairment in myocardial and cardiac function. Results. The images of 23 of the patients (82.1%) were positive. Two hundred and twenty-nine sub-segments with abnormal lesions were detected out of 391 cardiac sub-segments of these 23 positive cases. According to the segmental abnormalities, the cases were divided into two cases (8.7%) with single abnormal wall segment, six cases (26.1%) with two abnormal wall segments, and 15 cases (65.2%) with three or more abnormal wall segments or scattered lesions. Conclusions. 99Tcm-MIBI G-MPI can objectively show impaired myocardium in patients with sarcoglycanopathy. Therefore, this method is helpful for early diagnosis and follow-up of myocardial damage.
- 30 August 2019
- Published by VM Media Group sp. z o.o in Neurologia i Neurochirurgia Polska
- Vol. 53 (4) , 265-270
Clinical value of 99Tcm-MIBI gated myocardial perfusion imaging in evaluating sarcoglycanopathy
Aim. The purpose of this study was to analyse the diagnostic value of gated myocardial perfusion imaging (G-MPI) in the evaluation of myocardial injury in sarcoglycanopathy. Materials and methods. Twenty-eight patients diagnosed with sarcoglycanopathy were evaluated using 99m- -methoxyisobutylisonitrile(99Tcm-MIBI) G-MPI. The data was processed into tomographic images, and the left ventricular function was analysed using quantitative gated SPECT (QGS) to assess the degree of impairment in myocardial and cardiac function. Results. The images of 23 of the patients (82.1%) were positive. Two hundred and twenty-nine sub-segments with abnormal lesions were detected out of 391 cardiac sub-segments of these 23 positive cases. According to the segmental abnormalities, the cases were divided into two cases (8.7%) with single abnormal wall segment, six cases (26.1%) with two abnormal wall segments, and 15 cases (65.2%) with three or more abnormal wall segments or scattered lesions. Conclusions. 99Tcm-MIBI G-MPI can objectively show impaired myocardium in patients with sarcoglycanopathy. Therefore, this method is helpful for early diagnosis and follow-up of myocardial damage. Aim. The purpose of this study was to analyse the diagnostic value of gated myocardial perfusion imaging (G-MPI) in the evaluation of myocardial injury in sarcoglycanopathy. Materials and methods. Twenty-eight patients diagnosed with sarcoglycanopathy were evaluated using 99m- -methoxyisobutylisonitrile(99Tcm-MIBI) G-MPI. The data was processed into tomographic images, and the left ventricular function was analysed using quantitative gated SPECT (QGS) to assess the degree of impairment in myocardial and cardiac function. Results. The images of 23 of the patients (82.1%) were positive. Two hundred and twenty-nine sub-segments with abnormal lesions were detected out of 391 cardiac sub-segments of these 23 positive cases. According to the segmental abnormalities, the cases were divided into two cases (8.7%) with single abnormal wall segment, six cases (26.1%) with two abnormal wall segments, and 15 cases (65.2%) with three or more abnormal wall segments or scattered lesions. Conclusions. 99Tcm-MIBI G-MPI can objectively show impaired myocardium in patients with sarcoglycanopathy. Therefore, this method is helpful for early diagnosis and follow-up of myocardial damage.
- 13 December 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 27 (3) , 559-571
Mixed-Bandwidth Cross-Channel Speech Recognition via Joint Optimization of DNN-Based Bandwidth Expansion and Acoustic Modeling
Automatic speech recognition (ASR) systems are often built using scene related speech data due to large variations of transmission channels and sampling rates in different scenarios. In this study, we propose a general framework that establishes a unified model for diversified speech data with different sampling rates and channels. The framework is a joint optimization of deep neural network (DNN)-based bandwidth expansion and acoustic modeling to exploit a large amount of diversified training data. First, we design two novel DNN architectures to map the acoustic features from narrowband to wideband speech through direct mapping and progressive mapping. The learning targets of the direct mapping DNN (DNN-DM) are the acoustic features extracted from speech with the largest bandwidth, while the acoustic features from speech with all the other bandwidths are used as input. A progressive stacking network (PSN) gradually maps the features from the low sampling rates to the highest sampling rate through the design of intermediate target layers via multitask training. Then, in addition to these bandwidth expansion networks, we investigate several joint training strategies for DNN-based acoustic models. Our experiments conducted on three diversified large-scale Mandarin speech datasets with different recording channels and sampling rates (6, 8, and 16 kHz) show that the proposed unified model using PSN for bandwidth expansion not only is a more flexible and compact design than conventional multiple acoustic models with each bandwidth for a specific sampling rate, but also yields consistent and significant improvements over bandwidth-dependent models with an average relative word error rate reduction of 6.2%, indicating that the proposed model can fully utilize the diversified cross-channel speech data with multiple bandwidths. Moreover, the proposed methods are verified to be robust on different realistic scenes and can be effectively extended to a long short-term memory framework.
- 13 December 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 27 (3) , 559-571
Mixed-Bandwidth Cross-Channel Speech Recognition via Joint Optimization of DNN-Based Bandwidth Expansion and Acoustic Modeling
Automatic speech recognition (ASR) systems are often built using scene related speech data due to large variations of transmission channels and sampling rates in different scenarios. In this study, we propose a general framework that establishes a unified model for diversified speech data with different sampling rates and channels. The framework is a joint optimization of deep neural network (DNN)-based bandwidth expansion and acoustic modeling to exploit a large amount of diversified training data. First, we design two novel DNN architectures to map the acoustic features from narrowband to wideband speech through direct mapping and progressive mapping. The learning targets of the direct mapping DNN (DNN-DM) are the acoustic features extracted from speech with the largest bandwidth, while the acoustic features from speech with all the other bandwidths are used as input. A progressive stacking network (PSN) gradually maps the features from the low sampling rates to the highest sampling rate through the design of intermediate target layers via multitask training. Then, in addition to these bandwidth expansion networks, we investigate several joint training strategies for DNN-based acoustic models. Our experiments conducted on three diversified large-scale Mandarin speech datasets with different recording channels and sampling rates (6, 8, and 16 kHz) show that the proposed unified model using PSN for bandwidth expansion not only is a more flexible and compact design than conventional multiple acoustic models with each bandwidth for a specific sampling rate, but also yields consistent and significant improvements over bandwidth-dependent models with an average relative word error rate reduction of 6.2%, indicating that the proposed model can fully utilize the diversified cross-channel speech data with multiple bandwidths. Moreover, the proposed methods are verified to be robust on different realistic scenes and can be effectively extended to a long short-term memory framework.