Validation of an Automated Scoring Algorithm That Assesses Eye Exploration in a 3-Dimensional Virtual Reality Environment Using Eye-Tracking Sensors
Open Access
Abstract
Eye-tracking studies in virtual reality (VR) deliver insights into behavioral function. The gold standard of evaluating gaze behavior is based on manual scoring, which is labor-intensive. Previously proposed automated eye-tracking algorithms for VR head mount display (HMD) were not validated against manual scoring, or tested in dynamic areas of interest (AOIs). Our study validates the accuracy of an automated scoring algorithm, which determines temporal fixation behavior on static and dynamic AOIs in VR, against subjective human annotation. The interclass-correlation coefficient (ICC) was calculated for the time of first fixation (TOFF) and total fixation duration (TFD), in ten participants, each presented with 36 static and dynamic AOIs. High ICC values (≥0.982; p < 0.0001) were obtained when comparing the algorithm-generated TOFF and TFD to the raters’ annotations. In sum, our algorithm is accurate in determining temporal parameters related to gaze behavior when using HMD-based VR. Thus, the significant time required for human scoring among numerous raters can be rendered obsolete with a reliable automated scoring system. The algorithm proposed here was designed to sub-serve a separate study that uses TOFF and TFD to differentiate apathy from depression in those suffering from Alzheimer’s dementia.Keywords
Funding Information
- ADDF Diagnostics Accelerator (GDADB 201906-2018926)
This publication has 31 references indexed in Scilit:
- A novel method for objective quantification of apathy based on gaze and physiological reactivity to stimuli presented in a virtual reality environmentAlzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring, 2025
- Eye movements to absent objects during mental imagery and visual memory in immersive virtual realityVirtual Reality, 2020
- Eye tracking – The overlooked method to measure cognition in neurodegeneration?Neuropsychologia, 2019
- The influence of gender and body dissatisfaction on body‐related attentional bias: An eye‐tracking and virtual reality studyInternational Journal of Eating Disorders, 2019
- Assessing cognitive functioning in females with Rett syndrome by eye-tracking methodologyEuropean Journal of Paediatric Neurology, 2018
- Validation of eye‐tracking measures of social attention as a potential biomarker for autism clinical trialsAutism Research, 2017
- Beyond eye gaze: What else can eyetracking reveal about cognition and cognitive development?Developmental Cognitive Neuroscience, 2017
- A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability ResearchJournal of Chiropractic Medicine, 2016
- Areas of Interest as a Signal Detection Problem in Behavioral Eye‐Tracking ResearchJournal of Behavioral Decision Making, 2015
- A confidence interval approach to sample size estimation for interobserver agreement studies with multiple raters and outcomesJournal of Clinical Epidemiology, 2012