Hao Shi
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Hao Shi has a total of 13 co-authors. Here are the top co-authors by number of shared publications:
- Tatsuya Kawahara
Graduate School of Informatics, Kyoto University, Kyoto, Japan
4 shared publications - Xugang Lu
National Institute of Information and Communications Technology, Kyoto, Japan
2 shared publications - Yuan Gao
Kyoto University
2 shared publications - Chenhui Chu
Kyoto University
1 shared publication - Cuicui Zhang
Tianjin University
1 shared publication - Hao Wei
Tianjin Key Laboratory for Marine Environmental Research and Service School of Marine Science and Technology Tianjin University Tianjin China
1 shared publication
Latest Publications
- Open Access
- 26 November 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 17 (1) , 829-840
Bridging Speech Emotion Recognition and Personality: Dataset and Temporal Interaction Condition Network
This study investigates the interaction between personality traits and emotion expression, exploring how personality information can improve speech emotion recognition (SER). We collect the personality annotation for the IEMOCAP dataset, making it the first speech dataset that contains both emotion and personality annotations (PA-IEMOCAP), and enabling direct integration of personality traits into SER. Statistical analysis on this dataset identified significant correlations between per sonality traits and emotional expressions. To extract finegrained personality features, we propose a temporal interaction condition network (TICN), in which personality features are integrated with HuBERT-based acoustic features for SER. Experiments show that incorporating ground-truth personality traits significantly enhances valence recognition, improving the concordance correlation coefficient (CCC) from 0.698 to 0.785 compared to the baseline without personality information. For practical applications in dialogue systems where personality information about the user is unavailable, we develop a front-end module of automatic personality recognition. Using these automatically predicted traits as inputs to our proposed TICN model, we achieve a CCC of 0.776 for valence recognition, representing an 11.17% relative improvement over the baseline. These findings confirm the effectiveness of personality-aware SER and provide a solid foundation for further exploration in personality-aware speech processing applications.
- 26 November 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 17 (1) , 829-840
Open AccessBridging Speech Emotion Recognition and Personality: Dataset and Temporal Interaction Condition Network
This study investigates the interaction between personality traits and emotion expression, exploring how personality information can improve speech emotion recognition (SER). We collect the personality annotation for the IEMOCAP dataset, making it the first speech dataset that contains both emotion and personality annotations (PA-IEMOCAP), and enabling direct integration of personality traits into SER. Statistical analysis on this dataset identified significant correlations between per sonality traits and emotional expressions. To extract finegrained personality features, we propose a temporal interaction condition network (TICN), in which personality features are integrated with HuBERT-based acoustic features for SER. Experiments show that incorporating ground-truth personality traits significantly enhances valence recognition, improving the concordance correlation coefficient (CCC) from 0.698 to 0.785 compared to the baseline without personality information. For practical applications in dialogue systems where personality information about the user is unavailable, we develop a front-end module of automatic personality recognition. Using these automatically predicted traits as inputs to our proposed TICN model, we achieve a CCC of 0.776 for valence recognition, representing an 11.17% relative improvement over the baseline. These findings confirm the effectiveness of personality-aware SER and provide a solid foundation for further exploration in personality-aware speech processing applications.
- Open Access
- 9 October 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech and Language Processing
- Vol. 33, 4253-4266
Combining Deterministic Enhanced Conditions With Dual-Streaming Encoding for Diffusion-Based Speech Enhancement
Score-based diffusion models for speech enhancement (SE) need to incorporate correct prior knowledge as reliable conditions to generate accurate predictions. However, providing reliable conditions using noisy features is challenging. One solution is to use features enhanced by deterministic methods as conditions. However, the information distortion and loss caused by deterministic methods might affect the diffusion process. In this paper, we first investigate the effects of using different deterministic SE models as conditions for diffusion. We validate two conditions depending on whether the noisy feature was used as part of the condition: one using only the deterministic feature (deterministic-only), and the other using both deterministic and noisy features (deterministic-noisy). Preliminary investigation found that using deterministic enhanced conditions improves DNSMOS and UTMOS on real data, while the choice between using deterministic-only or deterministic-noisy conditions depends on the deterministic models. Based on these findings, we propose the deterministic-diffusion unified model for SE to more effectively utilize both conditions. Moreover, we found that fine-grained deterministic models have greater potential in objective evaluation metrics, while UNet-based deterministic models provide more stable diffusion performance. Therefore, we also propose a deterministic model that combines coarse- and fine-grained processing for the diffusion. Experimental results on CHiME4 show that the proposed models effectively leverage deterministic models to achieve better SE evaluation scores on the DNSMOS and UTMOS for real evaluation sets. In addition, the proposed deterministic model proves to be more stable than other deterministic models when it is used for diffusion.
- 9 October 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech and Language Processing
- Vol. 33, 4253-4266
Open AccessCombining Deterministic Enhanced Conditions With Dual-Streaming Encoding for Diffusion-Based Speech Enhancement
Score-based diffusion models for speech enhancement (SE) need to incorporate correct prior knowledge as reliable conditions to generate accurate predictions. However, providing reliable conditions using noisy features is challenging. One solution is to use features enhanced by deterministic methods as conditions. However, the information distortion and loss caused by deterministic methods might affect the diffusion process. In this paper, we first investigate the effects of using different deterministic SE models as conditions for diffusion. We validate two conditions depending on whether the noisy feature was used as part of the condition: one using only the deterministic feature (deterministic-only), and the other using both deterministic and noisy features (deterministic-noisy). Preliminary investigation found that using deterministic enhanced conditions improves DNSMOS and UTMOS on real data, while the choice between using deterministic-only or deterministic-noisy conditions depends on the deterministic models. Based on these findings, we propose the deterministic-diffusion unified model for SE to more effectively utilize both conditions. Moreover, we found that fine-grained deterministic models have greater potential in objective evaluation metrics, while UNet-based deterministic models provide more stable diffusion performance. Therefore, we also propose a deterministic model that combines coarse- and fine-grained processing for the diffusion. Experimental results on CHiME4 show that the proposed models effectively leverage deterministic models to achieve better SE evaluation scores on the DNSMOS and UTMOS for real evaluation sets. In addition, the proposed deterministic model proves to be more stable than other deterministic models when it is used for diffusion.
- 30 May 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 32 (23299290) , 3049-3060
Waveform-Domain Speech Enhancement Using Spectrogram Encoding for Robust Speech Recognition
While waveform-domain speech enhancement (SE) has been extensively investigated in recent years and achieves state-of-the-art performance in many datasets, spectrogram-based SE tends to show robust and stable enhancement behavior. In this paper, we propose a waveform-spectrogram hybrid method (WaveSpecEnc) to improve the robustness of waveform-domain SE. WaveSpecEnc refines the corresponding temporal feature map by spectrogram encoding in each encoder layer. Incorporating spectral information provides robust human hearing experience performance. However, it has a minor automatic speech recognition (ASR) improvement. Thus, we improve it for robust ASR by further utilizing spectrogram encoding information (WaveSpecEnc+) to both the SE front-end and ASR back-end. Experimental results using the CHiME-4 dataset show that ASR performance in real evaluation sets is consistently improved with the proposed method, which outperformed others, including DEMUCS and Conv-Tasnet. Refining in the shallow encoder layers is very effective, and the effect is confirmed even with a strong ASR baseline using WavLM.
- 30 May 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 32 (23299290) , 3049-3060
Waveform-Domain Speech Enhancement Using Spectrogram Encoding for Robust Speech Recognition
While waveform-domain speech enhancement (SE) has been extensively investigated in recent years and achieves state-of-the-art performance in many datasets, spectrogram-based SE tends to show robust and stable enhancement behavior. In this paper, we propose a waveform-spectrogram hybrid method (WaveSpecEnc) to improve the robustness of waveform-domain SE. WaveSpecEnc refines the corresponding temporal feature map by spectrogram encoding in each encoder layer. Incorporating spectral information provides robust human hearing experience performance. However, it has a minor automatic speech recognition (ASR) improvement. Thus, we improve it for robust ASR by further utilizing spectrogram encoding information (WaveSpecEnc+) to both the SE front-end and ASR back-end. Experimental results using the CHiME-4 dataset show that ASR performance in real evaluation sets is consistently improved with the proposed method, which outperformed others, including DEMUCS and Conv-Tasnet. Refining in the shallow encoder layers is very effective, and the effect is confirmed even with a strong ASR baseline using WavLM.
- 17 June 2023
- Published by Springer Nature in Climate Dynamics
- Vol. 61 (11-12) , 5271-5286
Tendency-and-attention-informed deep learning for ENSO forecasts
- 17 June 2023
- Published by Springer Nature in Climate Dynamics
- Vol. 61 (11-12) , 5271-5286
Tendency-and-attention-informed deep learning for ENSO forecasts