Mohsen Abbaspour Onari

Technical and ethical concerns impede the establishment of trust among healthcare professionals (HCPs) in developing artificial intelligence (AI)-based decision support. Yet, our understanding of trust models is constrained, and a standard accepted approach to evaluating trust in AI models is still lacking. We introduce a novel methodology to assess and quantify HCPs’ perceived trust in an interpretable machine learning model that serves as clinical decision support for diagnosing COVID-19 cases. Our approach leverages fuzzy cognitive maps (FCMs) to elicit and quantify HCPs’ trust mental models for understanding trust dynamics in clinical diagnosis. Our study reveals that HCPs rely predominantly on their own expertise when interacting with the developed interpretable clinical decision support. Although the model’s interpretations offer limited assistance in diagnostic tasks, they facilitate the HCPs’ utilization of it. However, the impact of these interpretations on the establishment of perceived trust varies among HCPs, which can lead to an increase in trust for some while decreasing it for others. To validate quantified perceived trust, we employ the degree of agreement metric, which quantitatively assesses whether HCPs lean more towards their own expertise or rely on the model’s recommendations in diagnostic tasks. We found significant alignment between the conclusions of the two metrics, indicating successful modeling and quantification of perceived trust. Plus, a moderate to strong positive correlation between the two metrics confirmed this conclusion. This means that FCMs can quantify HCPs’ perceived trust, aligning with their actual diagnostic advice shift after interacting with the model.

The increasing use of artificial intelligence (AI) models across various fields has raised concerns about whether these models can meet user trust expectations. As a result, researchers are focusing on assessing AI models’ performance relative to user expectations to determine trust levels. Evidence suggests that effective interaction with eXplainable AI (XAI) techniques can mitigate over-reliance on AI models and better align user expectations with the actual capabilities of these models in decision-making. In this study, we analyze trust from two perspectives: perceived trust, based on user self-reported trust, and demonstrated trust, which evaluates whether users, when given a choice, prefer to rely on AI or make decisions independently. We also explore how different interactions between human subjects and XAI models, along with varying levels of task risk, influence trust. Our findings reveal that these two types of trust are substantially different; human subjects do not always exhibit trust behavior in actual decision-making tasks, even when they perceive themselves as trusting the AI. Furthermore, we show that an AI model’s low error rate in making correct decisions can influence human subjects’ mental models, leading them to report a higher tendency to trust the AI. Finally, we conclude that human perceptions of trust are fragile and may change based on ongoing interactions with the model.

Distal myopathy represents a genetically heterogeneous group of skeletal muscle disorders with broad clinical manifestations, posing diagnostic challenges in radiology. To address this, we propose a novel multimodal attention-aware fusion architecture that combines features extracted from two distinct deep learning models, one capturing global contextual information and the other focusing on local details, representing complementary aspects of the input data. Uniquely, our approach integrates these features through an attention gate mechanism, enhancing both predictive performance and interpretability. Our method achieves a high classification accuracy on the BUSI benchmark and a proprietary distal myopathy dataset, while also generating clinically relevant saliency maps that support transparent decision-making in medical diagnosis. We rigorously evaluated interpretability through (1) functionally grounded metrics, coherence scoring against reference masks and incremental deletion analysis, and (2) application-grounded validation with seven expert radiologists. While our fusion strategy boosts predictive performance relative to single-stream and alternative fusion strategies, both quantitative and qualitative evaluations reveal persistent gaps in anatomical specificity and clinical usefulness of the interpretability. These findings highlight the need for richer, context-aware interpretability methods and human-in-the-loop feedback to meet clinicians' expectations in real-world diagnostic settings.