A Multi-Kernel Embedding Fusion Framework for Physiological Signal Based Emotion Recognition
- 21 April 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 16 (3) , 2440-2453
Abstract
Physiological signal-based emotion recognition requires effective fusion of multi-modal physiological signals to improve recognition accuracy. In this paper, a multi-kernel embedding fusion framework (MKEFF) is proposed for multi-modal physiological signal emotion recognition. Specifically, multi-kernel learning and kernel approximation techniques are used to compute the multi-kernel embeddings of the original feature vectors of each modality independently. The embeddings are then fed in parallel to their respective representation learning layer, where the proposed sparse relation learning method is applied to all the modalities to explore the correlation and diversity among them. Finally, a distribution alignment based fusion method is proposed to align each modality in the subspace, and a weighted summation fusion is performed to obtain the fused representations. Extensive cross-subject emotion recognition experiments are conducted on three public datasets, DEAP, DECAF, and SEED-IV, to evaluate the proposed method. The experimental results demonstrate that the proposed method achieves better classification performance and interpretability than the state-of-the-art methods.Keywords
Funding Information
- National Key R&D Program of China (2022YFB4700200)
- National Natural Science Foundation of China (62333007)
- Shenzhen Science and Technology Program (JCYJ20220818103602004)
This publication has 39 references indexed in Scilit:
- Human emotion recognition from EEG-based brain–computer interface using machine learning: a comprehensive reviewNeural Computing and Applications, 2022
- Bridging deep and multiple kernel learning: A reviewInformation Fusion, 2021
- Physiological-signal-based emotion recognition: An odyssey from methodology to philosophyMeasurement, 2021
- A snapshot research and implementation of multimodal information fusion for data-driven emotion recognitionInformation Fusion, 2020
- Detecting driving stress in physiological signals based on multimodal feature analysis and kernel classifiersExpert Systems with Applications, 2017
- Ensemble application of convolutional neural networks and multiple kernel learning for multimodal sentiment analysisNeurocomputing, 2017
- A review of affective computing: From unimodal analysis to multimodal fusionInformation Fusion, 2017
- Learning deep kernels in the space of dot product polynomialsMachine Learning, 2016
- EasyMKL: a scalable multiple kernel learning algorithmNeurocomputing, 2015
- An iterative thresholding algorithm for linear inverse problems with a sparsity constraintCommunications on Pure and Applied Mathematics, 2004