When to Explain: Modeling User Need for Explanations in Real-World Autonomous Driving
Open Access
- 13 April 2026
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1-20
Abstract
The integration of artificial intelligence into autonomous vehicles (AVs) raises transparency challenges that can hinder user acceptance and experience. To address when users need AV explanations, we created a large-scale dataset of 3327 diverse driving scenarios, each paired with a user-friendly explanation, and conducted an online study to survey when users need explanations. Using both scenario and user-related factors, our best-performing tree-ensemble models predicted explanation need with great performance (F1=0.72, AUC=0.82). SHAP analyses revealed that while both user- and scenario-related factors matter, factors directly related to driving (AV driving style, AV action, event cause, annual mileage, human driving style) were more contributive than general demographics and environmental factors. Our study delivers a comprehensively annotated dataset that underpins future human-AV interaction research, an explainable model that reliably predicts explanation needs, and valuable insights to inform the design of adaptive AV interfaces for superior user experiences.Keywords
This publication has 73 references indexed in Scilit:
- A Taxonomy of Explanation Types and Need Indicators in Human–Agent CollaborationsInternational Journal of Social Robotics, 2024
- The effect of transparency and trust on intelligent system acceptance: Evidence from a user-based studyElectronic Markets, 2022
- Regularized target encoding outperforms traditional methods in supervised machine learning with high cardinality featuresComputational Statistics, 2022
- In the Passenger Seat: Differences in the Perception of Human vs. Automated Vehicle Control and Resulting HMI Demands of Users2020
- Textual Explanations for Self-Driving Vehicles2018
- Comparison-Based Inverse Classification for Interpretability in Machine Learning2018
- Graph Geodesics to Find Progressively Similar Skin Lesion Images2017
- Why did my car just do that? Explaining semi-autonomous driving actions to improve driver understanding, trust, and performanceInternational Journal on Interactive Design and Manufacturing (IJIDeM), 2014
- Handbook of Driver Assistance Systems2014
- An introduction to case-based reasoningArtificial Intelligence Review, 1992