Group Synchrony for Emotion Recognition Using Physiological Signals
- 7 April 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 14 (4) , 2614-2625
Abstract
During group interactions, we react and modulate our emotions and behaviour to the group through phenomena including emotion contagion and physiological synchrony. Previous work on emotion recognition through video/image has shown that group context information improves the classification performance. However, when using physiological data, literature mostly focuses on intrapersonal models that leave-out group information, while interpersonal models are unexplored. This paper introduces a new interpersonal Weighted Group Synchrony approach, which relies on Electrodermal Activity (EDA) and Heart-Rate Variability (HRV). We perform an analysis of synchrony metrics applied across diverse data representations (EDA and HRV morphology and features, recurrence plot, spectrogram), to identify which metrics and modalities better characterise physiological synchrony for emotion recognition. We explored two datasets (AMIGOS and K-EmoCon), covering different group sizes (4 vs dyad) and group-based activities (video-watching vs conversation). The experimental results show that integrating group information improves arousal and valence classification, across all datasets, with the exception of K-EmoCon on valence. The proposed method was able to attain mean M-F1 of $\approx$ 72.15% arousal and 81.16% valence for AMIGOS, and M-F1 of $\approx$ 52.63% arousal, 65.09% valence for K-EmoCon, surpassing previous work results for K-EmoCon on arousal, and providing a new baseline on AMIGOS for long-videos.Keywords
This publication has 63 references indexed in Scilit:
- Systolic Peak Detection in Acceleration Photoplethysmograms Measured from Emergency Responders in Tropical ConditionsPLOS ONE, 2013
- Towards a Theory of Collective EmotionsEmotion Review, 2013
- Physiological compliance for social gaming analysis: Cooperative versus competitive playInteracting with Computers, 2012
- Cardiac and Respiratory Patterns Synchronize between Persons during Choir SingingPLOS ONE, 2011
- Synchronized arousal between performers and related spectators in a fire-walking ritualProceedings of the National Academy of Sciences, 2011
- Theories of emotion causation: A reviewCognition and Emotion, 2009
- Measures of emotion: A reviewCognition and Emotion, 2009
- Social coordination dynamics: Measuring human bondingSocial Neuroscience, 2008
- THE PSYCHOLOGICAL AND PHYSIOLOGICAL DYNAMICS OF A SIMPLE CONVERSATIONSocial Behavior and Personality: an international journal, 2004
- Interpersonal relationships and personal space: Research review and theoretical modelHuman Ecology, 1976