A Screen-Based Multimodal Virtual Classroom Interface for Understanding Behavioral Sensory Responses in Autistic Adolescents: A Pilot Study
Open Access
- 9 October 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Systems and Rehabilitation Engineering
- Vol. 33 (15344320) , 4067-4077
Abstract
Autism impacts at least 1 in 100 children worldwide, with about 90% experiencing sensory processing difficulties. Virtual Reality (VR), which can precisely deliver controlled sensory stimuli, has emerged as a promising tool for studying sensory experiences. However, VR systems using a head-mounted display may cause discomfort and exacerbate sensory challenges for autistic children. Screen-based VR could offer a viable alternative, but research on designing multimodal sensory delivery systems that simulate real-life experiences remains limited. As a result, the impact of on-screen VR on children’s behavioral sensory responses is not well understood. Therefore, as a pilot study to fill this gap, we designed a novel screen-based Multimodal Virtual Classroom Interface (MVCI) system. MVCI was designed to deliver well-controlled visual, auditory, and tactile stimuli that closely mimic a real classroom environment. The pilot study involved 9 autistic adolescents and 17 typically developing (TD) adolescents, all of whom reported 100% acceptance of the system. Quantitative behavioral analysis demonstrated that, even with the small sample size, the on-screen interaction showed significant differences (p < 0.05) between the two groups in terms of eye gaze, fine motor movements, and eye-hand alignment. Additionally, several behavioral patterns were strongly correlated with participants’ sensory profiles and ADHD symptom severity (p < 0.05, rs > 0.7). Using a novel Fixation Sequence Modeling (FSM) framework, we were able to predict participants’ near-future performance with high accuracy (97-98% proximity) based on their granular behavioral responses.Keywords
This publication has 41 references indexed in Scilit:
- User Experience Evaluation in Virtual Reality for Autism: A Systematic Literature Review2023
- Global prevalence of autism: A systematic review updateAutism Research, 2022
- A Non-immersive Virtual Reality Application for Children with Autism Spectrum Disorder2021
- Utilizing social virtual reality robot (V2R) for music education to children with high-functioning autismEducation and Information Technologies, 2021
- Head-Mounted Display-Based Virtual Reality as a Tool to Reduce Disruptive Behavior in a Student Diagnosed with Autism Spectrum Disorder2020
- Impact of mainstream classroom setting on attention of children with autism spectrum disorder: an eye-tracking studyUniversal Access in the Information Society, 2020
- A review on the long short-term memory modelArtificial Intelligence Review, 2020
- An immersive virtual reality educational intervention on people with autism spectrum disorders (ASD) for the development of communication skills and problem solvingEducation and Information Technologies, 2019
- Eye-Hand Coordination in Children with High Functioning Autism and Asperger’s Disorder Using a Gap-Overlap ParadigmJournal of Autism and Developmental Disorders, 2012
- Sample size of 12 per group rule of thumb for a pilot studyPharmaceutical Statistics, 2005