MSP-GEO Corpus: A Multimodal Database for Understanding Video-Learning Experience
Open Access
- 4 November 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 488-497
Abstract
Video-based learning has become a popular, scalable, and effective approach for students to learn new skills. Many of the challenges for video-based learning can be addressed with machine learning models. However, the available datasets often lack the rich source of data that is needed to accurately predict students’ learning experiences and outcomes. To address this limitation, we introduce the MSP-GEO corpus, a new multimodal database that contains detailed demographic and educational data, recordings of the students and their screens, and meta-data about the lecture during the learning experience. The MSP-GEO corpus was collected using a quasi-experimental pre-test/post-test design. It consists of more than 39,600 seconds (11 hours) of continuous facial footage from 76 participants watching one of three experimental videos on the topic of fossil formation, resulting in over one million facial images. The data collected includes 21 gaze synchronization points, webcam and monitor recordings, and metadata for pauses, plays, and timeline navigation. Additionally, we annotated the recordings for engagement, boredom, and confusion using human evaluators. The MSP-GEO corpus has the potential to improve the accuracy of video-based learning outcomes and experience predictions, facilitate research on the psychological processes of video-based learning, inform the design of instructional videos, and advance the development of learning analytics methods.Keywords
This publication has 32 references indexed in Scilit:
- Workflow for designing instructional videos to support place-based geoscience education for geoscience majorsJournal of Geoscience Education, 2022
- Recognition of human emotions using EEG signals: A reviewComputers in Biology and Medicine, 2021
- The REDCap consortium: Building an international community of software platform partnersJournal of Biomedical Informatics, 2019
- Prediction and Localization of Student Engagement in the Wild2018
- Calibration free, user-independent gaze estimation with tensor analysisImage and Vision Computing, 2018
- A survey of affective brain computer interfaces: principles, state-of-the-art, and challengesBrain-Computer Interfaces, 2014
- Analysis and Compensation of the Reaction Lag of Evaluators in Continuous Emotional Annotations2013
- Introducing the RECOLA multimodal corpus of remote collaborative and affective interactions2013
- Cognitive Theory of Multimedia Learning2005
- Coefficient Alpha and the Internal Structure of TestsPsychometrika, 1951