Implementing an Artificial Intelligence Decision Support System in Radiology: Prospective Qualitative Evaluation Study Using the Nonadoption Abandonment Scale-Up, Spread, and Sustainability (NASSS) Framework (Preprint)
Preprint
- 9 July 2025
- preprint
- Published by JMIR Publications Inc. in JMIR Preprints
Abstract
BACKGROUND Medical imaging remains at the forefront of advancements in adopting digital health technologies in clinical practice. Regulator-approved artificial intelligence (AI) clinical decision support systems are commercially available and being embedded into routine practices for radiologists internationally. These decision support solutions show promising clinical validity compared to standard practice conditions; however, their implementation over time and implications on radiologists’ practice are poorly understood. OBJECTIVE This paper aims to examine the real-world implementation of an AI clinical decision support tool in radiology through a qualitative evaluation across pre-, peri-, and postimplementation phases. Specifically, it seeks to identify the key contextual, organizational, and human factors shaping adoption and sustainability, to map these influences using the nonadoption, abandonment, scale-up, spread, and sustainability (NASSS) framework, and to generate insights that inform evidence-based strategies and policy for integrating AI safely and effectively into public hospital imaging services. METHODS This prospective study was conducted in a large public tertiary referral hospital in Brisbane, Queensland, Australia. One-to-one participant interviews were undertaken across the 3 implementation phases. Participants comprised radiology consultants, registrars, and radiographers involved in chest computed tomography studies during the study period. Interviews were guided by the NASSS framework to identify contextual factors influencing implementation. RESULTS A total of 43 semistructured interviews were conducted across baseline (n=16), peri-implementation (n=9), and postimplementation (n=18) phases, comprising 7 (16%) radiographers, 20 (47%) registrar radiologists, and 16 (37%) consultant radiologists. Across NASSS domains, 56 barriers and 18 enablers were identified at baseline, 55 and 14 during peri-implementation, and 82 and 33 postimplementation. Organizational barriers dominated early phases, while technological issues such as system accuracy, interoperability, and information overload became most prominent during and after rollout. Enablers increased over time, particularly within the technology and value proposition domains, as some clinicians adapted the AI as a secondary safety check. Trust and adoption remained constrained by performance inconsistency, weak communication, and medicolegal uncertainty. CONCLUSIONS The implementation of AI decision support in radiology is as much an organizational and cultural process as a technological one. Clinicians remain willing to engage, but sustainable adoption depends on consolidating early positive experiences and addressing negative ones, embedding communication and training, and maintaining iterative feedback between users, vendors, and system leaders. Applying the NASSS framework revealed how domains interact dynamically across time, offering both theoretical insight into sociotechnical complexity and practical guidance for hospitals seeking to move from pilot to routine, trustworthy AI integration.Keywords
All Related Versions
- Published version: Journal of Medical Internet Research, 28, e80342.
This publication has 25 references indexed in Scilit:
- Stakeholder engagement: the path to trustworthy AI in healthcare2025
- RETRACTED ARTICLE: Unleashing the power of advanced technologies for revolutionary medical imaging: pioneering the healthcare frontier with artificial intelligenceDiscover Artificial Intelligence, 2024
- A sociotechnical system perspective on AIMinds and Machines, 2024
- Patient Safety in Radiology and Medical Imaging2023
- Trust and stakeholder perspectives on the implementation of AI tools in clinical radiologyEuropean Radiology, 2023
- Radiology artificial intelligence: a systematic review and evaluation of methods (RAISE)European Radiology, 2022
- An international survey on AI in radiology in 1041 radiologists and radiology residents part 2: expectations, hurdles to implementation, and educationEuropean Radiology, 2021
- Artificial intelligence performance in detecting tumor metastasis from medical radiology imaging: A systematic review and meta-analysiseClinicalMedicine, 2021
- Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factorsEuropean Radiology, 2020
- Augmented Radiologist Workflow Improves Report Value and Saves Time: A Potential Model for Implementation of Artificial IntelligenceAcademic Radiology, 2020