Ercheng Pei
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Ercheng Pei has a total of 35 co-authors. Here are the top co-authors by number of shared publications:
- Dongmei Jiang
Pengcheng Laboratory, Shenzhen, China
6 shared publications - Hichem Sahli
Department of Electronics and Informatics, Vrije Universiteit Brussel, Brussels, Belgium
4 shared publications - Lang He
School of Computer Science and Technology, Xian University of Posts and Telecommunications, Xian, China
4 shared publications - Meshia Cedric Oveneke
Department ETRO, Vrije Universiteit Brussel (VUB), Brussels, Belgium
3 shared publications - Yong Zhao
Zhejiang Lab, Hangzhou, China
3 shared publications Abel Díaz Berenguer
Department of Electronics and Informatics (ETRO) Vrije Universiteit Brussel Brussels Belgium
2 shared publications
Latest Publications
- 18 March 2026
- Published by Elsevier in Pattern Recognition
- Vol. 179
LLM-driven fine-grained emotion parsing and parameterized mapping for conversational TTS
- 18 March 2026
- Published by Elsevier in Pattern Recognition
- Vol. 179
LLM-driven fine-grained emotion parsing and parameterized mapping for conversational TTS
- 13 February 2026
- Published by Elsevier in Information Fusion
- Vol. 132
FDA-CAPMA: Federated domain adaptation with co-activation pattern and multimodal mamba for fMRI depression detection
- 13 February 2026
- Published by Elsevier in Information Fusion
- Vol. 132
FDA-CAPMA: Federated domain adaptation with co-activation pattern and multimodal mamba for fMRI depression detection
- 17 December 2025
- Published by Elsevier in Information Fusion
- Vol. 129
Multi-modal prompt learning for facial expression recognition: Leveraging emojis and large language models
- 17 December 2025
- Published by Elsevier in Information Fusion
- Vol. 129
Multi-modal prompt learning for facial expression recognition: Leveraging emojis and large language models
- 5 October 2025
- Published by Elsevier in Knowledge-Based Systems
- Vol. 330
Classifier ensemble based source-free domain adaptation for time series classification
- 5 October 2025
- Published by Elsevier in Knowledge-Based Systems
- Vol. 330
Classifier ensemble based source-free domain adaptation for time series classification
- Open Access
- 1 January 2025
- Published by Institution of Engineering and Technology (IET) in IET Intelligent Transport Systems
- Vol. 19 (1)
An Efficient Illumination‐Invariant Dynamic Facial Expression Recognition for Driving Scenarios
Facial expression recognition (FER) is significant in many application scenarios, such as driving scenarios with very different lighting conditions between day and night. Existing methods primarily focus on eliminating the negative effects of pose and identity information on FER, but overlook the challenges posed by lighting variations. So, this work proposes an efficient illumination‐invariant dynamic FER method. To augment the robustness of FER methods to illumination variance, contrast normalisation is introduced to form a low‐level illumination‐invariant expression features learningmodule. In addition, to extract dynamic and salient expression features, a two‐stage temporal attention mechanism is introduced to form a high‐level dynamic salient expression features learning module deciphering dynamic facial expression patterns. Furthermore, additive angular margin loss is incorporated into the training of the proposed model to increase the distances between samples of different categories while reducing the distances between samples belonging to the same category. We conducted comprehensive experiments using the Oulu‐CASIA and DFEW datasets. On the Oulu‐CASIA VIS and NIR subsets in the normal illumination, the proposed method achieved accuracies of 92.08% and 91.46%, which are 1.01 and 7.06 percentage points higher than the SOTA results from the DCBLSTM and CELDL method, respectively. Based on the Oulu‐CASIA NIR subset in the dark illumination, the proposed method achieved an accuracies of 91.25%, which are 4.54 percentage points higher than the SOTA result from the CDLLNet method. On the DFEW dataset, the proposed method achieved a UAR of 60.67% and a WAR of 71.48% with 12M parameters, approaching the SOTA result from the VideoMAE model with 86M parameters. The outcomes of our experiments validate the effectiveness of the proposed dynamic FER method, affirming its ability in addressing the challenges posed by diverse illumination conditions in driving scenarios.
- 1 January 2025
- Published by Institution of Engineering and Technology (IET) in IET Intelligent Transport Systems
- Vol. 19 (1)
Open AccessAn Efficient Illumination‐Invariant Dynamic Facial Expression Recognition for Driving Scenarios
Facial expression recognition (FER) is significant in many application scenarios, such as driving scenarios with very different lighting conditions between day and night. Existing methods primarily focus on eliminating the negative effects of pose and identity information on FER, but overlook the challenges posed by lighting variations. So, this work proposes an efficient illumination‐invariant dynamic FER method. To augment the robustness of FER methods to illumination variance, contrast normalisation is introduced to form a low‐level illumination‐invariant expression features learningmodule. In addition, to extract dynamic and salient expression features, a two‐stage temporal attention mechanism is introduced to form a high‐level dynamic salient expression features learning module deciphering dynamic facial expression patterns. Furthermore, additive angular margin loss is incorporated into the training of the proposed model to increase the distances between samples of different categories while reducing the distances between samples belonging to the same category. We conducted comprehensive experiments using the Oulu‐CASIA and DFEW datasets. On the Oulu‐CASIA VIS and NIR subsets in the normal illumination, the proposed method achieved accuracies of 92.08% and 91.46%, which are 1.01 and 7.06 percentage points higher than the SOTA results from the DCBLSTM and CELDL method, respectively. Based on the Oulu‐CASIA NIR subset in the dark illumination, the proposed method achieved an accuracies of 91.25%, which are 4.54 percentage points higher than the SOTA result from the CDLLNet method. On the DFEW dataset, the proposed method achieved a UAR of 60.67% and a WAR of 71.48% with 12M parameters, approaching the SOTA result from the VideoMAE model with 86M parameters. The outcomes of our experiments validate the effectiveness of the proposed dynamic FER method, affirming its ability in addressing the challenges posed by diverse illumination conditions in driving scenarios.
- 31 August 2023
- Published by Elsevier in Expert Systems with Applications
- Vol. 236
An ensemble learning-enhanced multitask learning method for continuous affect recognition from facial images
- 31 August 2023
- Published by Elsevier in Expert Systems with Applications
- Vol. 236
An ensemble learning-enhanced multitask learning method for continuous affect recognition from facial images
- 1 April 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 25 (15209210) , 3709-3722
A Bayesian Filtering Framework for Continuous Affect Recognition From Facial Images
Continuous affective state estimation from facial information is a task which requires the prediction of time series of emotional state outputs from a facial image sequence. Modeling the spatial-temporal evolution of facial information plays an important role in affective state estimation. One of the most widely used methods is Recurrent Neural Networks (RNN). RNNs provide an attractive framework for propagating information over a sequence using a continuous-valued hidden layer representation. In this work, we propose to instead learn rich affective state dynamics. We model human affect as a dynamical system and define the affective state in terms of valence, arousal and their higher-order derivatives. We then pose the affective state estimation problem as a jointly trained state estimator for high-dimensional input images, combining an RNN and a Bayesian Filter, i.e. Kalman filters (KF) and Extended Kalman filters (EKF), so that all weights in the resulting network can be trained using backpropagation. We use a recently proposed general framework for designing and learning discriminative state estimators framed as computational graphs. Such approach can handle high dimensional observations and efficiently optimize, in an end-to-end fashion, the state estimator. In addition, to deal with the asynchrony between emotion labels and input images, caused by the inherent reaction lag of the annotators, we introduce a convolutional layer that aligns features with emotion labels. Experimental results, on the RECOLA and SEMAINE datasets for continuous emotion prediction, illustrate the potential of the proposed framework compared to recent state-of-the-art models.
- 1 April 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 25 (15209210) , 3709-3722
A Bayesian Filtering Framework for Continuous Affect Recognition From Facial Images
Continuous affective state estimation from facial information is a task which requires the prediction of time series of emotional state outputs from a facial image sequence. Modeling the spatial-temporal evolution of facial information plays an important role in affective state estimation. One of the most widely used methods is Recurrent Neural Networks (RNN). RNNs provide an attractive framework for propagating information over a sequence using a continuous-valued hidden layer representation. In this work, we propose to instead learn rich affective state dynamics. We model human affect as a dynamical system and define the affective state in terms of valence, arousal and their higher-order derivatives. We then pose the affective state estimation problem as a jointly trained state estimator for high-dimensional input images, combining an RNN and a Bayesian Filter, i.e. Kalman filters (KF) and Extended Kalman filters (EKF), so that all weights in the resulting network can be trained using backpropagation. We use a recently proposed general framework for designing and learning discriminative state estimators framed as computational graphs. Such approach can handle high dimensional observations and efficiently optimize, in an end-to-end fashion, the state estimator. In addition, to deal with the asynchrony between emotion labels and input images, caused by the inherent reaction lag of the annotators, we introduce a convolutional layer that aligns features with emotion labels. Experimental results, on the RECOLA and SEMAINE datasets for continuous emotion prediction, illustrate the potential of the proposed framework compared to recent state-of-the-art models.
- 17 March 2021
- Published by Wiley in Computer Graphics Forum
- Vol. 40 (6) , 47-61
Action Unit Driven Facial Expression Synthesis from a Single Image with Patch Attentive GAN
Recent advances in generative adversarial networks (GANs) have shown tremendous success for facial expression generation tasks. However, generating vivid and expressive facial expressions at Action Units (AUs) level is still challenging, due to the fact that automatic facial expression analysis for AU intensity itself is an unsolved difficult task. In this paper, we propose a novel synthesis‐by‐analysis approach by leveraging the power of GAN framework and state‐of‐the‐art AU detection model to achieve better results for AU‐driven facial expression generation. Specifically, we design a novel discriminator architecture by modifying the patch‐attentive AU detection network for AU intensity estimation and combine it with a global image encoder for adversarial learning to force the generator to produce more expressive and realistic facial images. We also introduce a balanced sampling approach to alleviate the imbalanced learning problem for AU synthesis. Extensive experimental results on DISFA and DISFA+ show that our approach outperforms the state‐of‐the‐art in terms of photo‐realism and expressiveness of the facial expression quantitatively and qualitatively.
- 17 March 2021
- Published by Wiley in Computer Graphics Forum
- Vol. 40 (6) , 47-61
Action Unit Driven Facial Expression Synthesis from a Single Image with Patch Attentive GAN
Recent advances in generative adversarial networks (GANs) have shown tremendous success for facial expression generation tasks. However, generating vivid and expressive facial expressions at Action Units (AUs) level is still challenging, due to the fact that automatic facial expression analysis for AU intensity itself is an unsolved difficult task. In this paper, we propose a novel synthesis‐by‐analysis approach by leveraging the power of GAN framework and state‐of‐the‐art AU detection model to achieve better results for AU‐driven facial expression generation. Specifically, we design a novel discriminator architecture by modifying the patch‐attentive AU detection network for AU intensity estimation and combine it with a global image encoder for adversarial learning to force the generator to produce more expressive and realistic facial images. We also introduce a balanced sampling approach to alleviate the imbalanced learning problem for AU synthesis. Extensive experimental results on DISFA and DISFA+ show that our approach outperforms the state‐of‐the‐art in terms of photo‐realism and expressiveness of the facial expression quantitatively and qualitatively.
- 25 September 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 23 (15209210) , 3540-3550
Monocular 3D Facial Expression Features for Continuous Affect Recognition
Automated facial expression analysis from image sequences for continuous emotion recognition is a very challenging task due to the loss of the three-dimensional information during the image formation process. State-of-the-art relied on estimating dynamic textures features and convolutional neural network features to derive spatio-temporal features. Despite their great success, such features are insensitive to micro facial muscle deformations and are affected by identity, face pose, illumination variation, and self-occlusion. In this work, we argue that retrieving, from image sequences, 3D facial spatio-temporal information, which describes the natural facial muscle deformation, provides a semantical and efficient way of representation and is useful for emotion recognition. In this paper, we propose a framework for extracting three-dimensional facial spatio-temporal features from monocular image sequences using an extended 3D Morphable Model (3DMM) which disentangles the identity factor from the facial expressions of a specific person. An LSTM model is used to evaluate the effectiveness of the proposed spatio-temporal features on video-based facial expression recognition task and continuous affect recognition task. Experimental results, on the AFEW6.0 datasets for facial expression recognition, and the RECOLA and SEMAINE datasets for continuous emotion prediction, illustrate the potential of the proposed 3D spatio-temporal features for facial expressions analysis and continuous affect recognition, as well as their efficiency compared to recent state-of-the-art features.
- 25 September 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 23 (15209210) , 3540-3550
Monocular 3D Facial Expression Features for Continuous Affect Recognition
Automated facial expression analysis from image sequences for continuous emotion recognition is a very challenging task due to the loss of the three-dimensional information during the image formation process. State-of-the-art relied on estimating dynamic textures features and convolutional neural network features to derive spatio-temporal features. Despite their great success, such features are insensitive to micro facial muscle deformations and are affected by identity, face pose, illumination variation, and self-occlusion. In this work, we argue that retrieving, from image sequences, 3D facial spatio-temporal information, which describes the natural facial muscle deformation, provides a semantical and efficient way of representation and is useful for emotion recognition. In this paper, we propose a framework for extracting three-dimensional facial spatio-temporal features from monocular image sequences using an extended 3D Morphable Model (3DMM) which disentangles the identity factor from the facial expressions of a specific person. An LSTM model is used to evaluate the effectiveness of the proposed spatio-temporal features on video-based facial expression recognition task and continuous affect recognition task. Experimental results, on the AFEW6.0 datasets for facial expression recognition, and the RECOLA and SEMAINE datasets for continuous emotion prediction, illustrate the potential of the proposed 3D spatio-temporal features for facial expressions analysis and continuous affect recognition, as well as their efficiency compared to recent state-of-the-art features.
- 11 February 2019
- Published by Springer Nature in Multimedia Tools and Applications
- Vol. 78 (14) , 19387-19412
Continuous affect recognition with weakly supervised learning
- 11 February 2019
- Published by Springer Nature in Multimedia Tools and Applications
- Vol. 78 (14) , 19387-19412
Continuous affect recognition with weakly supervised learning