Multimodal Fast–Slow Neural Network for learning engagement evaluation
- 3 February 2023
- journal article
- research article
- Published by Emerald Publishing in Data Technologies and Applications
- Vol. 57 (3) , 418-435
Abstract
Purpose: Student engagement is a key factor that connects with student achievement and retention. This paper aims to identify individuals' engagement automatically in the classroom with multimodal data for supporting educational research.Design/methodology/approach: The video and electroencephalogram data of 36 undergraduates were collected to represent observable and internal information. Since different modal data have different granularity, this study proposed the Fast–Slow Neural Network (FSNN) to detect engagement through both observable and internal information, with an asynchrony structure to preserve the sequence information of data with different granularity.Findings: Experimental results show that the proposed algorithm can recognize engagement better than the traditional data fusion methods. The results are also analyzed to figure out the reasons for the better performance of the proposed FSNN.Originality/value: This study combined multimodal data from observable and internal aspects to improve the accuracy of engagement detection in the classroom. The proposed FSNN used the asynchronous process to deal with the problem of remaining sequential information when facing multimodal data with different granularity.Keywords
This publication has 55 references indexed in Scilit:
- Representation Learning: A Review and New PerspectivesIEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
- Multimedia classification and event detection using double fusionMultimedia Tools and Applications, 2013
- Student engagement with school: Critical conceptual and methodological issues of the constructPsychology in the Schools, 2008
- A theoretical basis for standing and traveling brain waves measured with human EEG with implications for an integrated consciousnessClinical Neurophysiology, 2006
- Measuring cognitive and psychological engagement: Validation of the Student Engagement InstrumentJournal of School Psychology, 2006
- School Engagement: Potential of the Concept, State of the EvidenceReview of Educational Research, 2004
- Student Engagement in Instructional Activity: Patterns in the Elementary, Middle, and High School YearsAmerican Educational Research Journal, 2000
- Learning representations by back-propagating errorsNature, 1986
- Time-on-task: Issues of timing, sampling, and definition.Journal of Educational Psychology, 1982
- Dropout from Higher Education: A Theoretical Synthesis of Recent ResearchReview of Educational Research, 1975