Automatic Detection of Mind Wandering During Reading Using Gaze and Physiology
Open Access
- 9 November 2015
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 299-306
Abstract
Mind wandering (MW) entails an involuntary shift in attention from task-related thoughts to task-unrelated thoughts, and has been shown to have detrimental effects on performance in a number of contexts. This paper proposes an automated multimodal detector of MW using eye gaze and physiology (skin conductance and skin temperature) and aspects of the context (e.g., time on task, task difficulty). Data in the form of eye gaze and physiological signals were collected as 178 participants read four instructional texts from a computer interface. Participants periodically provided self-reports of MW in response to pseudorandom auditory probes during reading. Supervised machine learning models trained on features extracted from participants' gaze fixations, physiological signals, and contextual cues were used to detect pages where participants provided positive responses of MW to the auditory probes. Two methods of combining gaze and physiology features were explored. Feature level fusion entailed building a single model by combining feature vectors from individual modalities. Decision level fusion entailed building individual models for each modality and adjudicating amongst individual decisions. Feature level fusion resulted in an 11% improvement in classification accuracy over the best unimodal model, but there was no comparable improvement for decision level fusion. This was reflected by a small improvement in both precision and recall. An analysis of the features indicated that MW was associated with fewer and longer fixations and saccades, and a higher more deterministic skin temperature. Possible applications of the detector are discussed.Keywords
Funding Information
- NSF (1235958)
This publication has 37 references indexed in Scilit:
- Mind wandering while reading easy and difficult textsPsychonomic Bulletin & Review, 2013
- The eyes know what you are thinking: Eye movements as an objective measure of mind wanderingConsciousness and Cognition, 2011
- Meta-awareness, perceptual decoupling and the wandering mindTrends in Cognitive Sciences, 2011
- Pupillometric Evidence for the Decoupling of Attention from Perceptual Input during Offline ThoughtPLOS ONE, 2011
- A Wandering Mind Is an Unhappy MindScience, 2010
- The WEKA data mining softwareACM SIGKDD Explorations Newsletter, 2009
- Investigating the causes of wrap-up effects: Evidence from eye movements and E–Z ReaderCognition, 2009
- OGAMA (Open Gaze and Mouse Analyzer): Open-source software designed to analyze eye and mouse movements in slideshow study designsBehavior Research Methods, 2008
- Eye movements in reading and information processing: 20 years of research.Psychological Bulletin, 1998
- A Coefficient of Agreement for Nominal ScalesEducational and Psychological Measurement, 1960