Jinghan Wu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Jinghan Wu has a total of 13 co-authors. Here are the top co-authors by number of shared publications:
- Erwei Yin
Defense Innovation Institute, Academy of Military Sciences, Beijing, China
3 shared publications - Liang Xie
Defense Innovation Institute, Academy of Military Sciences, Beijing, China
3 shared publications - Yakun Zhang
Defense Innovation Institute, Academy of Military Sciences, Beijing, China
2 shared publications - Changyan Zheng
Defense Innovation Institute, Academy of Military Sciences, Beijing 100071, China
1 shared publication - Dong Ming
Tianjin University
1 shared publication - Dongliang Zhou
Tianjin University, Tianjin, China
1 shared publication
Latest Publications
- 14 July 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Human-Machine Systems
- Vol. 55 (4) , 559-568
AVE Speech: A Comprehensive Multimodal Dataset for Speech Recognition Integrating Audio, Visual, and Electromyographic Signals
The global aging population faces considerable challenges, particularly in communication, due to the prevalence of hearing and speech impairments. To address these, we introduce the AVE speech, a comprehensive multimodal dataset for speech recognition tasks. The dataset includes a 100-sentence Mandarin corpus with audio signals, lip-region video recordings, and six-channel electromyography data, collected from 100 participants. Each subject read the entire corpus ten times, with each sentence averaging approximately two seconds in duration, resulting in over 55 hours of multimodal speech data per modality. Experiments demonstrate that combining these modalities significantly improves recognition performance, particularly in cross-subject and high-noise environments. To our knowledge, this is the first publicly available sentence-level dataset integrating these three modalities for large-scale Mandarin speech recognition. We expect this dataset to drive advancements in both acoustic and nonacoustic speech recognition research, enhancing cross-modal learning and human–machine interaction.
- 14 July 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Human-Machine Systems
- Vol. 55 (4) , 559-568
AVE Speech: A Comprehensive Multimodal Dataset for Speech Recognition Integrating Audio, Visual, and Electromyographic Signals
The global aging population faces considerable challenges, particularly in communication, due to the prevalence of hearing and speech impairments. To address these, we introduce the AVE speech, a comprehensive multimodal dataset for speech recognition tasks. The dataset includes a 100-sentence Mandarin corpus with audio signals, lip-region video recordings, and six-channel electromyography data, collected from 100 participants. Each subject read the entire corpus ten times, with each sentence averaging approximately two seconds in duration, resulting in over 55 hours of multimodal speech data per modality. Experiments demonstrate that combining these modalities significantly improves recognition performance, particularly in cross-subject and high-noise environments. To our knowledge, this is the first publicly available sentence-level dataset integrating these three modalities for large-scale Mandarin speech recognition. We expect this dataset to drive advancements in both acoustic and nonacoustic speech recognition research, enhancing cross-modal learning and human–machine interaction.
- 2 July 2025
- Published by Elsevier in Neural Networks
- Vol. 191, 107783
MsDUNE: A multi-scale masked temporal fusion framework for speaker-independent lipreading via Dirichlet uncertainty estimation
- 2 July 2025
- Published by Elsevier in Neural Networks
- Vol. 191, 107783
MsDUNE: A multi-scale masked temporal fusion framework for speaker-independent lipreading via Dirichlet uncertainty estimation
- 18 September 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cognitive and Developmental Systems
- Vol. 15 (4) , 2282-2290
EMG-Based Cross-Subject Silent Speech Recognition Using Conditional Domain Adversarial Network
Machine learning techniques have achieved great success in electromyography (EMG) decoding, but EMG-based cross-subject silent speech recognition (SSR) received less attention because of its high individual variability. Therefore, this paper explores the field of cross-subject SSR to improve the recognition performance of EMG data collected from new subjects. Firstly, this paper reports on applying time-series features and one-dimensional convolutional neural networks (1D-CNN) for cross-subject SSR. Secondly, this paper proposes using a conditional domain adversarial network (CDAN) to solve the problem of reduced cross-subject SSR accuracy in the few samples’ datasets. It innovatively integrates the maximum mean difference (MMD) loss to get an improved condition domain adversarial network (ICDAN). While 1D-CNN is a feature extraction network that can meet the needs of cross subject silent speech recognition in large datasets, the recognition effect will be weakened in small datasets. Adding an ICDAN network after the feature extraction network can improve the problem of data distribution differences between the two domains, and further enhance recognition performance. The results show that the 1D-CNN model based on time-series features yields better results in the SSR of new subjects, and the ICDAN model can further improve the classification accuracy of cross-subjects in a few sample datasets by 14.88%.
- 18 September 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cognitive and Developmental Systems
- Vol. 15 (4) , 2282-2290
EMG-Based Cross-Subject Silent Speech Recognition Using Conditional Domain Adversarial Network
Machine learning techniques have achieved great success in electromyography (EMG) decoding, but EMG-based cross-subject silent speech recognition (SSR) received less attention because of its high individual variability. Therefore, this paper explores the field of cross-subject SSR to improve the recognition performance of EMG data collected from new subjects. Firstly, this paper reports on applying time-series features and one-dimensional convolutional neural networks (1D-CNN) for cross-subject SSR. Secondly, this paper proposes using a conditional domain adversarial network (CDAN) to solve the problem of reduced cross-subject SSR accuracy in the few samples’ datasets. It innovatively integrates the maximum mean difference (MMD) loss to get an improved condition domain adversarial network (ICDAN). While 1D-CNN is a feature extraction network that can meet the needs of cross subject silent speech recognition in large datasets, the recognition effect will be weakened in small datasets. Adding an ICDAN network after the feature extraction network can improve the problem of data distribution differences between the two domains, and further enhance recognition performance. The results show that the 1D-CNN model based on time-series features yields better results in the SSR of new subjects, and the ICDAN model can further improve the classification accuracy of cross-subjects in a few sample datasets by 14.88%.