The Impact of Expectation Management and Model Transparency on Radiologists’ Trust and Utilization of AI Recommendations for Lung Nodule Assessment on Computed Tomography: Simulated Use Study (Preprint)
Authors:
- Lotte J S EwalsLELotte J S Ewals
- Lynn J J HeesterbeekLHLynn J J Heesterbeek
- Bin YuBYBin Yu
- Kasper van der WulpKWKasper van der Wulp
- Dimitrios MavroeidisDMDimitrios Mavroeidis
- Mathias FunkMFMathias Funk
- Chris C P SnijdersCSChris C P Snijders
- Igor JacobsIJIgor Jacobs
- Joost NederendJNJoost Nederend
- Jon R PluyterJPJon R Pluyter
Preprint
- 27 August 2023
- preprint
- Published by JMIR Publications Inc. in JMIR Preprints
Abstract
BACKGROUND Many promising artificial intelligence (AI) and computer-aided detection and diagnosis systems have been developed, but few have been successfully integrated into clinical practice. This is partially owing to a lack of user-centered design of AI-based computer-aided detection or diagnosis (AI-CAD) systems. OBJECTIVE We aimed to assess the impact of different onboarding tutorials and levels of AI model explainability on radiologists’ trust in AI and the use of AI recommendations in lung nodule assessment on computed tomography (CT) scans. METHODS In total, 20 radiologists from 7 Dutch medical centers performed lung nodule assessment on CT scans under different conditions in a simulated use study as part of a 2×2 repeated-measures quasi-experimental design. Two types of AI onboarding tutorials (reflective vs informative) and 2 levels of AI output (black box vs explainable) were designed. The radiologists first received an onboarding tutorial that was either informative or reflective. Subsequently, each radiologist assessed 7 CT scans, first without AI recommendations. AI recommendations were shown to the radiologist, and they could adjust their initial assessment. Half of the participants received the recommendations via black box AI output and half received explainable AI output. Mental model and psychological trust were measured before onboarding, after onboarding, and after assessing the 7 CT scans. We recorded whether radiologists changed their assessment on found nodules, malignancy prediction, and follow-up advice for each CT assessment. In addition, we analyzed whether radiologists’ trust in their assessments had changed based on the AI recommendations. RESULTS Both variations of onboarding tutorials resulted in a significantly improved mental model of the AI-CAD system (informative P=.01 and reflective P=.01). After using AI-CAD, psychological trust significantly decreased for the group with explainable AI output (P=.02). On the basis of the AI recommendations, radiologists changed the number of reported nodules in 27 of 140 assessments, malignancy prediction in 32 of 140 assessments, and follow-up advice in 12 of 140 assessments. The changes were mostly an increased number of reported nodules, a higher estimated probability of malignancy, and earlier follow-up. The radiologists’ confidence in their found nodules changed in 82 of 140 assessments, in their estimated probability of malignancy in 50 of 140 assessments, and in their follow-up advice in 28 of 140 assessments. These changes were predominantly increases in confidence. The number of changed assessments and radiologists’ confidence did not significantly differ between the groups that received different onboarding tutorials and AI outputs. CONCLUSIONS Onboarding tutorials help radiologists gain a better understanding of AI-CAD and facilitate the formation of a correct mental model. If AI explanations do not consistently substantiate the probability of malignancy across patient cases, radiologists’ trust in the AI-CAD system can be impaired. Radiologists’ confidence in their assessments was improved by using the AI recommendations.Keywords
All Related Versions
- Published version: JMIR AI, 3 (1), e52211.
This publication has 41 references indexed in Scilit:
- A survey of computer-aided diagnosis of lung nodules from CT scans using deep learningComputers in Biology and Medicine, 2021
- Towards radiologist-level cancer risk assessment in CT lung screening using deep learningComputerized Medical Imaging and Graphics, 2021
- On the performance of lung nodule detection, segmentation and classificationComputerized Medical Imaging and Graphics, 2021
- Missed Lung CancerRadiologic Clinics of North America, 2018
- Improving the radiologist–CAD interaction: designing for appropriate trustClinical Radiology, 2015
- The moderating role of prior experience in technological acceptance models for ubiquitous computing services in urban environmentsTechnological Forecasting and Social Change, 2015
- Neural Networks2013
- Reflection and reflective practice in health professions education: a systematic reviewAdvances in Health Sciences Education, 2007
- Trust, self-confidence, and operators' adaptation to automationInternational Journal of Human-Computer Studies, 1994
- Trust between humans and machines, and the design of decision aidsInternational Journal of Man-Machine Studies, 1987