Multimodal Driver State Monitoring in a Simulated Environment: A Health-Oriented, Human-in-the-Loop Evaluation
Open Access
- 17 June 2026
- book chapter
- Published by Springer Nature
- p. 280-296
Abstract
Continuous monitoring of human cognitive and physiological state is a key challenge in digital health, particularly in safety-critical environments such as driving. This paper presents a multimodal driver-state monitoring framework integrating electroencephalography (EEG), wearable physiological sensing, and posture-based vision analysis. The proposed system enables synchronised, window-based analysis of cognitive workload and fatigue-related indicators across varying driving scenarios. We report preliminary results from a simulator-based pilot study, showing that the sensing modalities capture complementary aspects of driver state: EEG is sensitive to rapid cognitive changes, wearables reflect slower physiological trends related to fatigue and stress, and posture features capture behavioural adaptation. The study demonstrates the feasibility of multimodal integration while highlighting practical challenges, including sensor intrusiveness, movement-related noise, and inter-individual variability. Rather than proposing deployable algorithms, this work provides methodological insights to inform future real-vehicle implementations and illustrates how human-state awareness can be integrated into software-defined vehicle architectures to support health-aware, human-in-the-loop adaptation in future intelligent vehicles.Keywords
This publication has 17 references indexed in Scilit:
- Fatigue Detection with Multimodal Physiological Signals via Uncertainty-Aware Deep Transfer LearningJournal of Bionic Engineering, 2026
- Exploring the effect of driver drowsiness on takeover performance during automated driving: An updated literature reviewAccident Analysis & Prevention, 2025
- How to request drivers to prepare for takeovers during automated drivingTransportation Research Part F: Traffic Psychology and Behaviour, 2025
- Synchronization levels in EEG connectivity during cognitive workloads while drivingNonlinear Dynamics, 2024
- Driving simulator validation studies: A systematic reviewSimulation Modelling Practice and Theory, 2024
- Vehicle Computing2024
- Detecting driver fatigue using heart rate variability: A systematic reviewAccident Analysis & Prevention, 2022
- A definition of drowsiness: One purpose for sleep?Medical Hypotheses, 2008
- International standards on working postures and movements ISO 11226 and EN 1005-4Ergonomics, 2007
- Driver fatigue: Electroencephalography and psychological assessmentPsychophysiology, 2002