Fujiang Li
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Fujiang Li has a total of 12 co-authors. Here are the top co-authors by number of shared publications:
- Huizhi Liang
School of Computing, Newcastle University, Newcastle NE1 7RU, UK
3 shared publications - Shufei Duan
College of Computer Science and Technology, Shanxi University of Electronic Science and Technology, Linfen 041000, China
3 shared publications - Fangyu Zhao
College of Electronic Information Engineering, Taiyuan University of Technology, Jinzhong 030600, China
1 shared publication - Liangqi Li
College of Electronic Information Engineering, Taiyuan University of Technology, Jinzhong 030600, China
1 shared publication - Longhao Fu
College of Electronic Information Engineering, Taiyuan University of Technology, Jinzhong 030600, China
1 shared publication - Qianrong Guo
College of Electronic Information Engineering, Taiyuan University of Technology, Jinzhong 030600, China
1 shared publication
Latest Publications
- Open Access
- 2 April 2026
- Published by MDPI AG in Multimodal Technologies and Interaction
- Vol. 10 (4) , 38
PAD-Guided Multimodal Hybrid Contrastive Emotion Recognition upon STEM-E2VA Dataset
There are still challenges in speech emotion recognition, as the representation capability of single-modal information is limited, there are difficulties in capturing continuous emotional transitions in discrete emotion annotations, and the issues of modal structural differences and cross-sample alignment in multimodal fusion methods persist. To address these, this study undertakes work from both data and model perspectives. For data, a Chinese multimodal database STEM-E2VA was constructed, synchronously collecting four modalities of data: articulatory kinematics, acoustics, glottal signals, and videos. This covers seven discrete emotion categories and employs PAD continuous annotation. By integrating discrete and continuous dimensional annotations, it better represents the distinction between strong and weak emotions under the same discrete emotion label. Concurrently, to process the biases in PAD annotations, we employed the SCL-90 psychological questionnaire to analyze annotators’ cognitive and emotional perceptions, thereby ensuring data reliability. For model, this paper proposes a multimodal supervised contrastive fusion network incorporating PAD perception. It employs a PAD-enhanced hybrid contrastive loss function to optimize intra-model and inter-modal feature alignment. Utilizing a cross-attention mechanism combined with a GRU–Transformer network for temporal feature extraction, it achieves deep fusion of multimodal information, reducing inter-modal discrepancies and cross-class confusion. Experiments demonstrate that the proposed method achieves 85.47% accuracy in discrete sentiment recognition on STEM-E2VA, with a substantial reduction in RMSE for PAD dimension prediction. It also exhibits excellent generalization capability on IEMOCAP, providing a novel framework for integrating discrete and continuous sentiment representations.
- 2 April 2026
- Published by MDPI AG in Multimodal Technologies and Interaction
- Vol. 10 (4) , 38
Open AccessPAD-Guided Multimodal Hybrid Contrastive Emotion Recognition upon STEM-E2VA Dataset
There are still challenges in speech emotion recognition, as the representation capability of single-modal information is limited, there are difficulties in capturing continuous emotional transitions in discrete emotion annotations, and the issues of modal structural differences and cross-sample alignment in multimodal fusion methods persist. To address these, this study undertakes work from both data and model perspectives. For data, a Chinese multimodal database STEM-E2VA was constructed, synchronously collecting four modalities of data: articulatory kinematics, acoustics, glottal signals, and videos. This covers seven discrete emotion categories and employs PAD continuous annotation. By integrating discrete and continuous dimensional annotations, it better represents the distinction between strong and weak emotions under the same discrete emotion label. Concurrently, to process the biases in PAD annotations, we employed the SCL-90 psychological questionnaire to analyze annotators’ cognitive and emotional perceptions, thereby ensuring data reliability. For model, this paper proposes a multimodal supervised contrastive fusion network incorporating PAD perception. It employs a PAD-enhanced hybrid contrastive loss function to optimize intra-model and inter-modal feature alignment. Utilizing a cross-attention mechanism combined with a GRU–Transformer network for temporal feature extraction, it achieves deep fusion of multimodal information, reducing inter-modal discrepancies and cross-class confusion. Experiments demonstrate that the proposed method achieves 85.47% accuracy in discrete sentiment recognition on STEM-E2VA, with a substantial reduction in RMSE for PAD dimension prediction. It also exhibits excellent generalization capability on IEMOCAP, providing a novel framework for integrating discrete and continuous sentiment representations.
- Open Access
Facial Expression Annotation and Analytics for Dysarthria Severity Classification
Dysarthria in patients post-stroke is often accompanied by central facial paralysis, which impairs facial motor control and emotional expression. Current assessments rely on acoustic modalities, overlooking facial pathological cues and their correlation with emotional expression, which hinders comprehensive disease assessment. To address this issue, we propose a multimodal severity classification framework that integrates facial and acoustic features. Firstly, a multi-level annotation algorithm based on a pre-trained model and motion amplitude was designed to overcome the problem of data scarcity. Secondly, facial topology was modeled using Delaunay triangulation, with spatial relationships captured via graph convolutional networks (GCNs), while abnormal muscle coordination is quantified using facial action units (AUs). Finally, we proposed a multimodal feature set fusion technology framework to achieve the compensation of facial visual features for acoustic modalities and the analysis of disease classification. Our experimental results using the THE-POSSD dataset demonstrate an accuracy of 92.0% and an F1 score of 91.6%, significantly outperforming single-modality baselines. This study reveals the changes in facial movements and sensitive areas of patients under different emotional states, verifies the compensatory ability of visual patterns for auditory patterns, and demonstrates the potential of this multimodal framework for objective assessment and future clinical applications in speech disorders.
Open AccessFacial Expression Annotation and Analytics for Dysarthria Severity Classification
Dysarthria in patients post-stroke is often accompanied by central facial paralysis, which impairs facial motor control and emotional expression. Current assessments rely on acoustic modalities, overlooking facial pathological cues and their correlation with emotional expression, which hinders comprehensive disease assessment. To address this issue, we propose a multimodal severity classification framework that integrates facial and acoustic features. Firstly, a multi-level annotation algorithm based on a pre-trained model and motion amplitude was designed to overcome the problem of data scarcity. Secondly, facial topology was modeled using Delaunay triangulation, with spatial relationships captured via graph convolutional networks (GCNs), while abnormal muscle coordination is quantified using facial action units (AUs). Finally, we proposed a multimodal feature set fusion technology framework to achieve the compensation of facial visual features for acoustic modalities and the analysis of disease classification. Our experimental results using the THE-POSSD dataset demonstrate an accuracy of 92.0% and an F1 score of 91.6%, significantly outperforming single-modality baselines. This study reveals the changes in facial movements and sensitive areas of patients under different emotional states, verifies the compensatory ability of visual patterns for auditory patterns, and demonstrates the potential of this multimodal framework for objective assessment and future clinical applications in speech disorders.
- Open Access
- 22 December 2025
- Published by MDPI AG in Brain Sciences
- Vol. 16 (1) , 11
Lightweight EEG Phase Prediction Based on Channel Attention and Spatio-Temporal Parallel Processing
Background/Objectives: Closed-loop phase-locked TMS aims to deliver stimulation at targeted EEG phases, but real-time phase prediction remains a practical bottleneck. Timing errors are especially harmful near peaks and troughs, where small offsets can substantially degrade phase targeting. We benchmark representative predictors and develop models that improve phase consistency while reducing peak/trough lag. Methods: Using the publicly available Monash University TEPs–MEPs dataset, we benchmark classical predictors (AR- and FFT-based) and recurrent baselines (LSTM, GRU). To quantify extremum-specific behavior critical for closed-loop triggering, we propose Mean Lag Time (MLT), defined as the average temporal offset between predicted and ground-truth extrema, alongside PLV, APE, MAE, and RMSE. We further propose a parallel DSC-Attention-GRU architecture combining depthwise separable convolutions for efficient multi-channel spatio-temporal feature extraction with self-attention for spatial reweighting and dependency modeling, followed by a GRU phase predictor. A lightweight SqueezeNet-Attention-GRU variant is also designed for real-time constraints. Results: LSTM/GRU outperform AR/FFT in capturing temporal dynamics but retain residual peak/trough lag. Across stimulation intensities and frequency bands, DSC-Attention-GRU consistently improves phase consistency and prediction accuracy and reduces extremum lag, lowering MLT from ~7.77–7.79 ms to ~7.50–7.56 ms. The lightweight variant maintains stable performance with an average 3.7% inference speedup. Conclusions: Explicitly optimizing extremum timing via MLT and enhancing multi-channel modeling with DSC and attention reduces peak/trough lag and improves phase-consistent prediction, supporting low-latency closed-loop phase-locked TMS.
- 22 December 2025
- Published by MDPI AG in Brain Sciences
- Vol. 16 (1) , 11
Open AccessLightweight EEG Phase Prediction Based on Channel Attention and Spatio-Temporal Parallel Processing
Background/Objectives: Closed-loop phase-locked TMS aims to deliver stimulation at targeted EEG phases, but real-time phase prediction remains a practical bottleneck. Timing errors are especially harmful near peaks and troughs, where small offsets can substantially degrade phase targeting. We benchmark representative predictors and develop models that improve phase consistency while reducing peak/trough lag. Methods: Using the publicly available Monash University TEPs–MEPs dataset, we benchmark classical predictors (AR- and FFT-based) and recurrent baselines (LSTM, GRU). To quantify extremum-specific behavior critical for closed-loop triggering, we propose Mean Lag Time (MLT), defined as the average temporal offset between predicted and ground-truth extrema, alongside PLV, APE, MAE, and RMSE. We further propose a parallel DSC-Attention-GRU architecture combining depthwise separable convolutions for efficient multi-channel spatio-temporal feature extraction with self-attention for spatial reweighting and dependency modeling, followed by a GRU phase predictor. A lightweight SqueezeNet-Attention-GRU variant is also designed for real-time constraints. Results: LSTM/GRU outperform AR/FFT in capturing temporal dynamics but retain residual peak/trough lag. Across stimulation intensities and frequency bands, DSC-Attention-GRU consistently improves phase consistency and prediction accuracy and reduces extremum lag, lowering MLT from ~7.77–7.79 ms to ~7.50–7.56 ms. The lightweight variant maintains stable performance with an average 3.7% inference speedup. Conclusions: Explicitly optimizing extremum timing via MLT and enhancing multi-channel modeling with DSC and attention reduces peak/trough lag and improves phase-consistent prediction, supporting low-latency closed-loop phase-locked TMS.