Clinical Needs Assessment of a Machine Learning–Based Asthma Management Tool: User-Centered Design Approach
Open Access
- 15 January 2024
- journal article
- research article
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 8 (1) , e45391
Abstract
Background: Personalized asthma management depends on a clinician’s ability to efficiently review patient’s data and make timely clinical decisions. Unfortunately, efficient and effective review of these data is impeded by the varied format, location, and workflow of data acquisition, storage, and processing in the electronic health record. While machine learning (ML) and clinical decision support tools are well-positioned as potential solutions, the translation of such frameworks requires that barriers to implementation be addressed in the formative research stages. Objective: We aimed to use a structured user-centered design approach (double-diamond design framework) to (1) qualitatively explore clinicians’ experience with the current asthma management system, (2) identify user requirements to improve algorithm explainability and Asthma Guidance and Prediction System prototype, and (3) identify potential barriers to ML-based clinical decision support system use. Methods: At the “discovery” phase, we first shadowed to understand the practice context. Then, semistructured interviews were conducted digitally with 14 clinicians who encountered pediatric asthma patients at 2 outpatient facilities. Participants were asked about their current difficulties in gathering information for patients with pediatric asthma, their expectations of ideal workflows and tools, and suggestions on user-centered interfaces and features. At the “define” phase, a synthesis analysis was conducted to converge key results from interviewees’ insights into themes, eventually forming critical “how might we” research questions to guide model development and implementation. Results: We identified user requirements and potential barriers associated with three overarching themes: (1) usability and workflow aspects of the ML system, (2) user expectations and algorithm explainability, and (3) barriers to implementation in context. Even though the responsibilities and workflows vary among different roles, the core asthma-related information and functions they requested were highly cohesive, which allows for a shared information view of the tool. Clinicians hope to perceive the usability of the model with the ability to note patients’ high risks and take proactive actions to manage asthma efficiently and effectively. For optimal ML algorithm explainability, requirements included documentation to support the validity of algorithm development and output logic, and a request for increased transparency to build trust and validate how the algorithm arrived at the decision. Acceptability, adoption, and sustainability of the asthma management tool are implementation outcomes that are reliant on the proper design and training as suggested by participants. Conclusions: As part of our comprehensive informatics-based process centered on clinical usability, we approach the problem using a theoretical framework grounded in user experience research leveraging semistructured interviews. Our focus on meeting the needs of the practice with ML technology is emphasized by a user-centered approach to clinician engagement through upstream technology design.Keywords
All Related Versions
This publication has 20 references indexed in Scilit:
- Human-centred design processes for clinical decision support: A pulmonary embolism case studyInternational Journal of Medical Informatics, 2020
- Spaces for Participatory Design Innovation2020
- Prediction Models — Development, Evaluation, and Clinical ApplicationNew England Journal of Medicine, 2020
- Machine intelligence in healthcare—perspectives on trustworthiness, explainability, usability, and transparencynpj Digital Medicine, 2020
- Expert artificial intelligence-based natural language processing characterises childhood asthmaBMJ Open Respiratory Research, 2020
- A Clinician's Guide to Artificial Intelligence: How to Critically Appraise Machine Learning StudiesTranslational Vision Science & Technology, 2020
- How Participatory Design Works2019
- Clinical applications of machine learning algorithms: beyond the black boxBMJ, 2019
- Big Data in Medicine Is Driving Big ChangesYearbook of Medical Informatics, 2014
- A new methodology for formulating ill-structured problemsOmega, 1994