FEELing (key)Pressed: Implicit Touch Pressure Bests Brain Activity for Modeling Emotion Dynamics in the Space Between Stressed & Relaxed
- 1 July 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Haptics
- Vol. 17 (3) , 310-318
Abstract
In-body lived emotional experiences can be complex, with time-varying and dissonant emotions evolving simultaneously; devices responding in real-time to estimate personal human emotion should evolve accordingly. Models assuming generalized emotions exist as discrete states fail to operationalize valuable information inherent in the dynamic and individualistic nature of human emotions. Our multi-resolution emotion self-reporting procedure allows the construction of emotion labels along the Stressed-Relaxed scale, differentiating not only what the emotions are, but how they are transitioning - e.g., "hopeful but getting stressed" vs. "hopeful and starting to relax". We trained participant-dependent hierarchical models of contextualized individual experience to compare emotion classification by modality (brain activity and keypress force from a physical keyboard), then benchmarked classification performance at F1-scores = [0.44, 0.82] (chance F1=0.22, sigma = 0.01) and examined high-performing features. Notably, when classifying emotion evolution in the context of an experience that realistically varies in stress, pressure-based features from keypress force proved to be the more informative modality, and more convenient when considering intrusiveness and ease of collection and processing. Finally, we present our FEEL (Force, EEG and Emotion-Labelled) dataset, a collection of brain activity and keypress force data, labelled with self-reported emotion collected during tense videogame play (N = 16) and open-sourced for community exploration.Keywords
Funding Information
- Natural Sciences and Engineering Research Council of Canada
This publication has 49 references indexed in Scilit:
- Emotion Duration2021
- Hierarchical classification of data streams: a systematic literature reviewArtificial Intelligence Review, 2021
- Comparative efficacy of non-pharmacological interventions on agitation in people with dementia: A systematic review and Bayesian network meta-analysisInternational Journal of Nursing Studies, 2020
- Multimodal big data affective analytics: A comprehensive survey using text, audio, visual and physiological signalsJournal of Network and Computer Applications, 2020
- Emotional state classification from EEG data using machine learning approachNeurocomputing, 2014
- Interpretation of Social Touch on an Artificial Arm Covered with an EIT-based Sensitive SkinInternational Journal of Social Robotics, 2014
- Analysis of Electroencephalography (EEG) Signals and Its Categorization–A StudyProcedia Engineering, 2012
- The Role of Affective Touch in Human-Robot Interaction: Human Intent and Expectations in Touching the Haptic CreatureInternational Journal of Social Robotics, 2011
- A survey of hierarchical classification across different application domainsData Mining and Knowledge Discovery, 2010
- Measuring emotion: The self-assessment manikin and the semantic differentialJournal of Behavior Therapy and Experimental Psychiatry, 1994