Evelyn Vasquez

Highly automated vehicles often rely on takeover requests (TORs) that lack contextual transparency, treat drivers as passive fallbacks, and lead to poor situational awareness. To address this challenge, this study presents the Empowering Language Interaction (ELI) framework, a conversational interface powered by a large language model that supports bidirectional negotiation and collaborative human–AI teamwork. Using the CARLA driving simulator, 28 participants compared ELI with a conventional TOR baseline in both urban and peri-urban driving scenarios. The study employed a multidimensional evaluation approach, integrating telemetry data on driving performance with continuous monitoring of physiological indicators (electrocardiogram and electrodermal activity) and subjective questionnaires to assess driver trust and engagement. Results indicated that ELI sustained continuous driver engagement and improved the subjective comprehension of the vehicle’s state. Physiologically, the conversational interface maintained active cognitive load, preventing the abrupt autonomic spikes characteristic of traditional takeover requests. Furthermore, ELI outperformed the TOR baseline in safety metrics by reducing out-of-lane events and maintaining greater safety margins. Conversational interaction has shown potential to transform drivers from passive supervisors into active teammates, improving joint decision-making without inducing over-reliance on the automated system.

In the dynamic and complex environment of highly automated vehicles, ensuring driver safety is the most critical task. While automation promises to reduce human error, the driver’s role is shifting to that of a teammate who must remain vigilant and ready to intervene, making it essential to monitor their attention level. However, a significant challenge in this domain is the considerable inter-individual variability in how people physiologically respond to cognitive states, such as distraction. This study addresses this by developing a methodology that first groups drivers into distinct physiology-based clusters before training a predictive model. The study was conducted in a high-fidelity driving simulator, where multimodal data streams, including heart rate variability and electrodermal activity, were collected from 30 participants during conditional-automated driving experiments. Using a time-series k-means clustering algorithm, the researchers successfully partitioned the drivers into clusters based on their physiological and behavioral patterns, which did not correlate with demographic factors. Then, a Long Short-Term Memory model was trained for each cluster, which achieved similar predictive performance compared to a single, generalized model. This finding demonstrates that a personalized, cluster-based approach is feasible for physiology-based driver monitoring, providing a robust and replicable solution for developing accurate and reliable attention estimation systems.