Jun Chen
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Jun Chen has a total of 12 co-authors. Here are the top co-authors by number of shared publications:
- Zhiyong Wu
Tsinghua Shenzhen International Graduate School, Shenzhen, Guangdong, China
2 shared publications - Boshi Tang
Tsinghua University, Shenzhen, China
1 shared publication - Gang He
State Key Laboratory and Institute of Elemento-Organic Chemistry, College of Chemistry, Nankai University, Tianjin 300071, China
1 shared publication - Haiwei Xue
Tsinghua University, Beijing, China
1 shared publication - Helen Meng
Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong, Hong Kong, China
1 shared publication - Jinfeng Bai
Tomorrow Advancing Life
1 shared publication
Latest Publications
- Open Access
- 29 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 13-20
First-order Multi-label Learning with Cross-modal Interactions for Multimodal Emotion Recognition
Multimodal emotion recognition (MER) is essential for the machine to fully understand human intentions. Various deep neural network based models are proposed but it is still challenging to better model and fuse multimodal features. In addition, recent studies have focused on the classification task of predicting discrete labels, while lacking consideration of the dimension value. In this paper, we propose a multimodal fusion model based on Transformer architecture and cross-modal interactions, and adopt a multi-label learning algorithm of first-order strategy to predict discrete labels and dimension values respectively. We also propose a semi-supervised learning method of moment injection with unlabeled data to enhance the robustness of the model. Finally, we use ensemble learning to further improve the performance of the model. We evaluate the proposed method on the MER-MULTI sub-challenge of Multimodal Emotion Recognition Challenge (MER 2023). Experimental results demonstrate the promising performance of our proposed method, which can achieve the evaluation metric of 0.6765 on the test set.
- 29 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 13-20
Open AccessFirst-order Multi-label Learning with Cross-modal Interactions for Multimodal Emotion Recognition
Multimodal emotion recognition (MER) is essential for the machine to fully understand human intentions. Various deep neural network based models are proposed but it is still challenging to better model and fuse multimodal features. In addition, recent studies have focused on the classification task of predicting discrete labels, while lacking consideration of the dimension value. In this paper, we propose a multimodal fusion model based on Transformer architecture and cross-modal interactions, and adopt a multi-label learning algorithm of first-order strategy to predict discrete labels and dimension values respectively. We also propose a semi-supervised learning method of moment injection with unlabeled data to enhance the robustness of the model. Finally, we use ensemble learning to further improve the performance of the model. We evaluate the proposed method on the MER-MULTI sub-challenge of Multimodal Emotion Recognition Challenge (MER 2023). Experimental results demonstrate the promising performance of our proposed method, which can achieve the evaluation metric of 0.6765 on the test set.
- 4 June 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 34, 1-5
A Synthetic Corpus Generation Method for Neural Vocoder Training
Nowadays, neural vocoders are preferred for their ability to synthesize high-fidelity audio. However, training a neural vocoder requires a massive corpus of high-quality real audio, and the audio recording process is often labor-intensive. In this work, we propose a synthetic corpus generation method for neural vocoder training, which can easily generate synthetic audio with an unlimited number at nearly no cost. We explicitly model the prior characteristics of audio from multiple target domains simultaneously (e.g., speeches, singing voices, and instrumental pieces) to equip the generated audio data with these characteristics. And we show that our synthetic corpus allows the neural vocoder to achieve competitive results without any real audio in the training process. To validate the effectiveness of our proposed method, we performed empirical experiments on both speech and music utterances in subjective and objective metrics. The experimental results show that the neural vocoder trained with the synthetic corpus produced by our method can generalize to multiple target scenarios and has excellent singing voice (MOS: 4.20) and instrumental piece (MOS: 4.00) synthesis results.
- 4 June 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 34, 1-5
A Synthetic Corpus Generation Method for Neural Vocoder Training
Nowadays, neural vocoders are preferred for their ability to synthesize high-fidelity audio. However, training a neural vocoder requires a massive corpus of high-quality real audio, and the audio recording process is often labor-intensive. In this work, we propose a synthetic corpus generation method for neural vocoder training, which can easily generate synthetic audio with an unlimited number at nearly no cost. We explicitly model the prior characteristics of audio from multiple target domains simultaneously (e.g., speeches, singing voices, and instrumental pieces) to equip the generated audio data with these characteristics. And we show that our synthetic corpus allows the neural vocoder to achieve competitive results without any real audio in the training process. To validate the effectiveness of our proposed method, we performed empirical experiments on both speech and music utterances in subjective and objective metrics. The experimental results show that the neural vocoder trained with the synthetic corpus produced by our method can generalize to multiple target scenarios and has excellent singing voice (MOS: 4.20) and instrumental piece (MOS: 4.00) synthesis results.