Real-Time Prediction of Player Engagement from Multimodal Data
- 12 March 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. PP (15209210) , 1-13
Abstract
Predicting player engagement in video games offers significant advantages for game design and quality assessment. Current approaches for predicting engagement, however, are costly, invasive, and/or suffer from recall bias, which limits their employability in practice. This paper presents a new multimodal neural network model for engagement prediction and conducts systematic analyses to identify practical solutions for real-world applications. We develop a hybrid architecture combining convolutional downsampling with transformer-based sequence modeling that effectively processes temporal data from multiple sensing channels, including EEG, eye tracking, and webcam footage. Through comprehensive comparative analysis, we determine which modalities and specific features most strongly contribute to engagement detection. Our evaluation demonstrates that webcam-based approaches utilizing geometric facial features and deep learning-based emotional embeddings achieve performance comparable to specialized biometric equipment while requiring only standard hardware. Feature attribution analysis reveals that head pose dynamics and facial embeddings are the most influential engagement signals, with neurophysiological features showing limited contribution. Cross-modal analysis confirms that facial emotion-based combinations achieve the strongest performance. Statistical power analysis reveals participant variability, highlighting generalization challenges. These findings demonstrate that effective engagement measurement is achievable using accessible webcam technology, offering an optimal balance between accuracy and accessibility for widespread commercial implementation in gaming contexts.Keywords
Funding Information
- Natural Sciences and Engineering Research Council of Canada (ALLRP556311-20)
This publication has 47 references indexed in Scilit:
- Experience sampling methodology and technology: an approach for examining situational, longitudinal, and multi-dimensional characteristics of engagementEducational Technology Research and Development, 2023
- Assessing student engagement from facial behavior in on-line learningMultimedia Tools and Applications, 2022
- Rethinking (Dis)engagement in human-computer interactionComputers in Human Behavior, 2022
- Non-intrusive Measurement of Player Engagement and Emotions - Real-Time Deep Neural Network Analysis of Facial Expressions During Game Play2020
- Measuring Affective, Physiological and Behavioural Differences in Solo, Competitive and Collaborative Games2016
- Flow in the Context of Daily Experience Fluctuation2016
- Game Experience Questionnaire2013
- On the Measurement and Conceptualization of Flow2012
- Measuring and defining the experience of immersion in gamesInternational Journal of Human-Computer Studies, 2008
- What is user engagement? A conceptual framework for defining user engagement with technologyJournal of the American Society for Information Science and Technology, 2008