Implementing a Commercial AI Fracture Detection Tool in Health Care Using the Non-Adoption, Abandonment, Scale-Up, Spread, and Sustainability Framework: A Formative Evaluation Study
Open Access
- 26 June 2026
- journal article
- case report
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 10 (1) , e72648
Abstract
Background: Artificial intelligence (AI) has the potential to enhance resource efficiency, improve patient treatment, and increase safety in health care. Still, there is limited knowledge on how to implement and evaluate AI solutions in real-world clinical settings. To address this gap, we conducted a formative process evaluation of the first large-scale procurement and implementation of a commercial AI solution in Norwegian health care. F The Non-Adoption, Abandonment, Scale-up, Spread, and Sustainability (NASSS) framework, was used for the formative process evaluation throughout the 4-year project to guide data collection, analysis, and real-time feedback. Objective: This study aimed to evaluate the usefulness of the NASSS framework for formative process evaluation of AI implementation in health care. Methods: A formative process evaluation was conducted from 2020 to 2024, covering the procurement, preimplementation, and implementation phases. Data included 65 interviews, observations, and document analysis. Data were analyzed thematically using the 7 NASSS domains, supplemented with subtopics within each domain to capture emerging infrastructural complexities and temporal dynamics. Real-time findings were discussed with the implementation team, decision-makers, and clinicians. Results: Key factors for successful implementation included clinician trust, workflow integration, task distribution, and digital maturity. Major challenges comprised limited documentation of Conformité Européenne–marked solutions, deskilling, and misaligned financial incentives. The NASSS framework enabled the identification of sociotechnical values and complexities, but did not fully capture workflow evolution and changing user perceptions over time. Conclusions: The NASSS framework is useful for evaluating AI implementation but requires adaptation to capture temporal dynamics and workflow changes better. These findings contribute to improving evaluation approaches for AI in health care.Keywords
This publication has 25 references indexed in Scilit:
- The need to strengthen the evaluation of the impact of Artificial Intelligence-based decision support systems on healthcare provisionHealth Policy, 2023
- Formative independent evaluation of a digital change programme in the English National Health Service: study protocol for a longitudinal qualitative studyBMJ Open, 2020
- Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factorsEuropean Radiology, 2020
- The rise of artificial intelligence in healthcare applications2020
- Using formative evaluation methods to improve clinical implementation efforts: Description and an examplePsychiatry Research, 2020
- A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysisThe Lancet Digital Health, 2019
- Spreading and scaling up innovation and improvementBMJ, 2019
- Information Infrastructures within European Health Care2017
- Healthcare professionals’ organisational barriers to health information technologies—A literature reviewInternational Journal of Medical Informatics, 2011
- Design Theory for Dynamic Complexity in Information Infrastructures: The Case of Building InternetJournal of Information Technology, 2010