Automatic Detection of Mind Wandering from Video in the Lab and in the Classroom
- 2 April 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 12 (4) , 974-988
Abstract
We report two studies that used facial features to automatically detect mind wandering, a ubiquitous phenomenon whereby attention drifts from the current task to unrelated thoughts. In a laboratory study, university students $(N = 152)$ read a scientific text, whereas in a classroom study high school students $(N = 135)$ learned biology from an intelligent tutoring system. Mind wandering was measured using validated self-report methods. In the lab, we recorded face videos and analyzed these at six levels of granularity: (1) upper-body movement; (2) head pose; (3) facial textures; (4) facial action units (AUs); (5) co-occurring AUs; and (6) temporal dynamics of AUs. Due to privacy constraints, videos were not recorded in the classroom. Instead, we extracted head pose, AUs, and AU co-occurrences in real-time. Machine learning models, consisting of support vector machines (SVM) and deep neural networks, achieved F1 scores of .478 and .414 (25.4% and 20.9% above-chance improvements, both with SVMs) for detecting mind wandering in the lab and classroom, respectively. The lab-based detectors achieved 8.4% improvement over the previous state-of-the-art; no comparison is available for classroom detectors. We discuss how the detectors can integrate into intelligent interfaces to increase engagement and learning by responding to wandering minds.
Keywords
Funding Information
- National Science Foundation (DRL 1235958, IIS 1523091)
This publication has 95 references indexed in Scilit:
- Mind wandering in sentence reading: Decoupling the link between mind and eye.Canadian Journal of Experimental Psychology / Revue canadienne de psychologie expérimentale, 2013
- Drifting from slow to “d'oh!”: Working memory capacity and mind wandering predict extreme reaction times and executive control errors.Journal of Experimental Psychology: Learning, Memory, and Cognition, 2012
- The eyes know what you are thinking: Eye movements as an objective measure of mind wanderingConsciousness and Cognition, 2011
- Does mind wandering reflect executive function or executive failure? Comment on Smallwood and Schooler (2006) and Watkins (2008).Psychological Bulletin, 2010
- Experience sampling during fMRI reveals default network and executive system contributions to mind wanderingProceedings of the National Academy of Sciences, 2009
- Conducting the train of thought: Working memory capacity, goal neglect, and mind wandering in an executive-control task.Journal of Experimental Psychology: Learning, Memory, and Cognition, 2009
- Not proper ROC curves as new tool for the analysis of differentially expressed genes in microarray experimentsBMC Bioinformatics, 2008
- Mind-wandering and dysphoriaCognition and Emotion, 2007
- Support-vector networksMachine Learning, 1995
- Divergence measures based on the Shannon entropyIEEE Transactions on Information Theory, 1991