Shauna M. Overgaard
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Shauna M. Overgaard has a total of 74 co-authors. Here are the top co-authors by number of shared publications:
- Tracey A. Brereton
Mayo Clinic, Rochester, MN, USA
7 shared publications - Chung-Il Wi
Mayo Clinic, Rochester, MN, USA
6 shared publications - Momin M. Malik
Precision Population Science Lab, Mayo Clinic, Rochester, MN 55905, USA
6 shared publications - Lu Zheng
Center for Digital Health, Mayo Clinic, Rochester, MN, USA
5 shared publications - Young J. Juhn
Department of Pediatric and Adolescent Medicine, Mayo Clinic , Rochester, Minnesota
5 shared publications - Joshua W. Ohde
Center for Digital Health, Mayo Clinic, Rochester, MN, United States
4 shared publications
Latest Publications
- Open Access
Addressing Power of Equivalence Tests for Statistical Fairness Assessments of Machine Learning Models in Healthcare
Background/Objectives: Machine learning and artificial intelligence (AI) have substantial potential to change the practice of oncology through individualized patient care; however, it is important to ensure that any benefits are realized equitably. There are several examples of AI tools exhibiting bias towards traditionally under-resourced or marginalized groups. Proper validation that these models perform well across populations requires rigorous statistical design and interpretation. One component that is currently lacking in fairness assessments is a clear, prespecified definition of the evidence required to conclude that an AI tool is fair. Methods: To address this issue, we embed the notion of fairness assessment of an AI model within the statistical equivalence testing framework. This approach provides an a priori definition of fairness and an explicit decision rule for concluding, with a high degree of confidence, that model performance falls within a prespecified margin. The framework also supplies power and sample size calculations to support the design phase of a fairness assessment, which we develop for the case of modeling a binary outcome and a binary classification. As a demonstration, we use a “four-fifths” rule of thumb to set equivalence margins but emphasize the necessity of clinical and ethical context for choosing the margins in practice. Results: Using two published oncology models, we show that a realistic sample size can be adequate for some performance metrics and inadequate for others. We also address power and sample size considerations for asymmetric association measures and impossibility results for fairness. Conclusions: Assessments of AI performance fairness should be designed with intention such that target metrics and equivalence margins are specified in advance with ample sample size to conclude fairness.
Open AccessAddressing Power of Equivalence Tests for Statistical Fairness Assessments of Machine Learning Models in Healthcare
Background/Objectives: Machine learning and artificial intelligence (AI) have substantial potential to change the practice of oncology through individualized patient care; however, it is important to ensure that any benefits are realized equitably. There are several examples of AI tools exhibiting bias towards traditionally under-resourced or marginalized groups. Proper validation that these models perform well across populations requires rigorous statistical design and interpretation. One component that is currently lacking in fairness assessments is a clear, prespecified definition of the evidence required to conclude that an AI tool is fair. Methods: To address this issue, we embed the notion of fairness assessment of an AI model within the statistical equivalence testing framework. This approach provides an a priori definition of fairness and an explicit decision rule for concluding, with a high degree of confidence, that model performance falls within a prespecified margin. The framework also supplies power and sample size calculations to support the design phase of a fairness assessment, which we develop for the case of modeling a binary outcome and a binary classification. As a demonstration, we use a “four-fifths” rule of thumb to set equivalence margins but emphasize the necessity of clinical and ethical context for choosing the margins in practice. Results: Using two published oncology models, we show that a realistic sample size can be adequate for some performance metrics and inadequate for others. We also address power and sample size considerations for asymmetric association measures and impossibility results for fairness. Conclusions: Assessments of AI performance fairness should be designed with intention such that target metrics and equivalence margins are specified in advance with ample sample size to conclude fairness.
- Open Access
- 14 May 2026
- Published by Elsevier in Journal of the National Medical Association
- Vol. 118 (4) , 789-816
A lifecycle governance and learning health system framework for trustworthy, generalizable, and sustainable human-ai partnership in clinical practice: Lessons from the asthma-guidance and prediction system (A-GPS)
- 14 May 2026
- Published by Elsevier in Journal of the National Medical Association
- Vol. 118 (4) , 789-816
Open AccessA lifecycle governance and learning health system framework for trustworthy, generalizable, and sustainable human-ai partnership in clinical practice: Lessons from the asthma-guidance and prediction system (A-GPS)
- Open Access
Governing Decisions of Probability Cutoffs in Clinical AI Deployment: A Case Study of Asthma Exacerbation Prediction
Models that estimate the probability of an adverse clinical outcome require an operational cutoff to translate continuous estimated probabilities into discrete labels that can trigger clinical action. Although statistical methods identify ‘optimal’ cut-offs, threshold selection ultimately reflects value judgments regarding harm tolerance, resource allocation, and workflow feasibility. We describe a governance-informed approach to selecting a deployment threshold for an asthma exacerbation (AE) prediction model integrated into clinical workflows. Using prevalence-adjusted performance metrics and real-world provider capacity modeling, we evaluated multiple candidate thresholds and quantified downstream workload and missed-event trade-offs. We demonstrate that statistically optimal thresholds may produce operationally infeasible alert volumes or unacceptable miss rates. We propose a structured threshold governance framework integrating statistical performance, clinical utility, stakeholder input, and human oversight safeguards. This case illustrates how threshold decisions should be treated as organizational governance processes rather than purely technical optimizations.
Open AccessGoverning Decisions of Probability Cutoffs in Clinical AI Deployment: A Case Study of Asthma Exacerbation Prediction
Models that estimate the probability of an adverse clinical outcome require an operational cutoff to translate continuous estimated probabilities into discrete labels that can trigger clinical action. Although statistical methods identify ‘optimal’ cut-offs, threshold selection ultimately reflects value judgments regarding harm tolerance, resource allocation, and workflow feasibility. We describe a governance-informed approach to selecting a deployment threshold for an asthma exacerbation (AE) prediction model integrated into clinical workflows. Using prevalence-adjusted performance metrics and real-world provider capacity modeling, we evaluated multiple candidate thresholds and quantified downstream workload and missed-event trade-offs. We demonstrate that statistically optimal thresholds may produce operationally infeasible alert volumes or unacceptable miss rates. We propose a structured threshold governance framework integrating statistical performance, clinical utility, stakeholder input, and human oversight safeguards. This case illustrates how threshold decisions should be treated as organizational governance processes rather than purely technical optimizations.
- Open Access
AImedReport: A Prototype Tool to Facilitate Research Reporting and Translation of AI Technologies in Healthcare
AImedReport is a proof-of-concept team-based documentation strategy that consolidates available AI research reporting guidelines and centrally tracks and organizes any information provided as part of following various guidelines. It functions to assist teams by a) outlining phases of the AI lifecycle and clinical evaluation; b) iteratively developing a comprehensive documentation deliverable and historical archive; and c) addressing translation, implementation, and accountability gaps. By acting as a hub for determining what information to capture, it helps navigate team responsibilities, simplify compliance with evaluation and reporting measures, and fulfill requirements to support clinical trial documentation and publications. Here, we give an overview of this system and describe how it can be used to address documentation and collaboration challenges in AI translation.
Open AccessAImedReport: A Prototype Tool to Facilitate Research Reporting and Translation of AI Technologies in Healthcare
AImedReport is a proof-of-concept team-based documentation strategy that consolidates available AI research reporting guidelines and centrally tracks and organizes any information provided as part of following various guidelines. It functions to assist teams by a) outlining phases of the AI lifecycle and clinical evaluation; b) iteratively developing a comprehensive documentation deliverable and historical archive; and c) addressing translation, implementation, and accountability gaps. By acting as a hub for determining what information to capture, it helps navigate team responsibilities, simplify compliance with evaluation and reporting measures, and fulfill requirements to support clinical trial documentation and publications. Here, we give an overview of this system and describe how it can be used to address documentation and collaboration challenges in AI translation.
- Open Access
- 15 January 2024
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 8 (1) , e45391
Clinical Needs Assessment of a Machine Learning–Based Asthma Management Tool: User-Centered Design Approach
Background: Personalized asthma management depends on a clinician’s ability to efficiently review patient’s data and make timely clinical decisions. Unfortunately, efficient and effective review of these data is impeded by the varied format, location, and workflow of data acquisition, storage, and processing in the electronic health record. While machine learning (ML) and clinical decision support tools are well-positioned as potential solutions, the translation of such frameworks requires that barriers to implementation be addressed in the formative research stages. Objective: We aimed to use a structured user-centered design approach (double-diamond design framework) to (1) qualitatively explore clinicians’ experience with the current asthma management system, (2) identify user requirements to improve algorithm explainability and Asthma Guidance and Prediction System prototype, and (3) identify potential barriers to ML-based clinical decision support system use. Methods: At the “discovery” phase, we first shadowed to understand the practice context. Then, semistructured interviews were conducted digitally with 14 clinicians who encountered pediatric asthma patients at 2 outpatient facilities. Participants were asked about their current difficulties in gathering information for patients with pediatric asthma, their expectations of ideal workflows and tools, and suggestions on user-centered interfaces and features. At the “define” phase, a synthesis analysis was conducted to converge key results from interviewees’ insights into themes, eventually forming critical “how might we” research questions to guide model development and implementation. Results: We identified user requirements and potential barriers associated with three overarching themes: (1) usability and workflow aspects of the ML system, (2) user expectations and algorithm explainability, and (3) barriers to implementation in context. Even though the responsibilities and workflows vary among different roles, the core asthma-related information and functions they requested were highly cohesive, which allows for a shared information view of the tool. Clinicians hope to perceive the usability of the model with the ability to note patients’ high risks and take proactive actions to manage asthma efficiently and effectively. For optimal ML algorithm explainability, requirements included documentation to support the validity of algorithm development and output logic, and a request for increased transparency to build trust and validate how the algorithm arrived at the decision. Acceptability, adoption, and sustainability of the asthma management tool are implementation outcomes that are reliant on the proper design and training as suggested by participants. Conclusions: As part of our comprehensive informatics-based process centered on clinical usability, we approach the problem using a theoretical framework grounded in user experience research leveraging semistructured interviews. Our focus on meeting the needs of the practice with ML technology is emphasized by a user-centered approach to clinician engagement through upstream technology design.
- 15 January 2024
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 8 (1) , e45391
Open AccessClinical Needs Assessment of a Machine Learning–Based Asthma Management Tool: User-Centered Design Approach
Background: Personalized asthma management depends on a clinician’s ability to efficiently review patient’s data and make timely clinical decisions. Unfortunately, efficient and effective review of these data is impeded by the varied format, location, and workflow of data acquisition, storage, and processing in the electronic health record. While machine learning (ML) and clinical decision support tools are well-positioned as potential solutions, the translation of such frameworks requires that barriers to implementation be addressed in the formative research stages. Objective: We aimed to use a structured user-centered design approach (double-diamond design framework) to (1) qualitatively explore clinicians’ experience with the current asthma management system, (2) identify user requirements to improve algorithm explainability and Asthma Guidance and Prediction System prototype, and (3) identify potential barriers to ML-based clinical decision support system use. Methods: At the “discovery” phase, we first shadowed to understand the practice context. Then, semistructured interviews were conducted digitally with 14 clinicians who encountered pediatric asthma patients at 2 outpatient facilities. Participants were asked about their current difficulties in gathering information for patients with pediatric asthma, their expectations of ideal workflows and tools, and suggestions on user-centered interfaces and features. At the “define” phase, a synthesis analysis was conducted to converge key results from interviewees’ insights into themes, eventually forming critical “how might we” research questions to guide model development and implementation. Results: We identified user requirements and potential barriers associated with three overarching themes: (1) usability and workflow aspects of the ML system, (2) user expectations and algorithm explainability, and (3) barriers to implementation in context. Even though the responsibilities and workflows vary among different roles, the core asthma-related information and functions they requested were highly cohesive, which allows for a shared information view of the tool. Clinicians hope to perceive the usability of the model with the ability to note patients’ high risks and take proactive actions to manage asthma efficiently and effectively. For optimal ML algorithm explainability, requirements included documentation to support the validity of algorithm development and output logic, and a request for increased transparency to build trust and validate how the algorithm arrived at the decision. Acceptability, adoption, and sustainability of the asthma management tool are implementation outcomes that are reliant on the proper design and training as suggested by participants. Conclusions: As part of our comprehensive informatics-based process centered on clinical usability, we approach the problem using a theoretical framework grounded in user experience research leveraging semistructured interviews. Our focus on meeting the needs of the practice with ML technology is emphasized by a user-centered approach to clinician engagement through upstream technology design.
- Open Access
- 25 November 2023
- Published by Springer Nature in npj Digital Medicine
- Vol. 6 (1) , 1-5
Implementing quality management systems to close the AI translation gap and facilitate safe, ethical, and effective health AI solutions
The integration of Quality Management System (QMS) principles into the life cycle of development, deployment, and utilization of machine learning (ML) and artificial intelligence (AI) technologies within healthcare settings holds the potential to close the AI translation gap by establishing a robust framework that accelerates the safe, ethical, and effective delivery of AI/ML in day-to-day patient care. Healthcare organizations (HCOs) can implement these principles effectively by embracing an enterprise QMS analogous to those in regulated industries. By establishing a QMS explicitly tailored to health AI technologies, HCOs can comply with evolving regulations and minimize redundancy and rework while aligning their internal governance practices with their steadfast commitment to scientific rigor and medical excellence.
- 25 November 2023
- Published by Springer Nature in npj Digital Medicine
- Vol. 6 (1) , 1-5
Open AccessImplementing quality management systems to close the AI translation gap and facilitate safe, ethical, and effective health AI solutions
The integration of Quality Management System (QMS) principles into the life cycle of development, deployment, and utilization of machine learning (ML) and artificial intelligence (AI) technologies within healthcare settings holds the potential to close the AI translation gap by establishing a robust framework that accelerates the safe, ethical, and effective delivery of AI/ML in day-to-day patient care. Healthcare organizations (HCOs) can implement these principles effectively by embracing an enterprise QMS analogous to those in regulated industries. By establishing a QMS explicitly tailored to health AI technologies, HCOs can comply with evolving regulations and minimize redundancy and rework while aligning their internal governance practices with their steadfast commitment to scientific rigor and medical excellence.
- Open Access
- 14 July 2023
- Published by JMIR Publications Inc. in Interactive Journal of Medical Research
- Vol. 12 (1) , e45903
The Role of Artificial Intelligence Model Documentation in Translational Science: Scoping Review
Background: Despite the touted potential of artificial intelligence (AI) and machine learning (ML) to revolutionize health care, clinical decision support tools, herein referred to as medical modeling software (MMS), have yet to realize the anticipated benefits. One proposed obstacle is the acknowledged gaps in AI translation. These gaps stem partly from the fragmentation of processes and resources to support MMS transparent documentation. Consequently, the absence of transparent reporting hinders the provision of evidence to support the implementation of MMS in clinical practice, thereby serving as a substantial barrier to the successful translation of software from research settings to clinical practice. Objective: This study aimed to scope the current landscape of AI- and ML-based MMS documentation practices and elucidate the function of documentation in facilitating the translation of ethical and explainable MMS into clinical workflows. Methods: A scoping review was conducted in accordance with PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines. PubMed was searched using Medical Subject Headings key concepts of AI, ML, ethical considerations, and explainability to identify publications detailing AI- and ML-based MMS documentation, in addition to snowball sampling of selected reference lists. To include the possibility of implicit documentation practices not explicitly labeled as such, we did not use documentation as a key concept but as an inclusion criterion. A 2-stage screening process (title and abstract screening and full-text review) was conducted by 1 author. A data extraction template was used to record publication-related information; barriers to developing ethical and explainable MMS; available standards, regulations, frameworks, or governance strategies related to documentation; and recommendations for documentation for papers that met the inclusion criteria. Results: Of the 115 papers retrieved, 21 (18.3%) papers met the requirements for inclusion. Ethics and explainability were investigated in the context of AI- and ML-based MMS documentation and translation. Data detailing the current state and challenges and recommendations for future studies were synthesized. Notable themes defining the current state and challenges that required thorough review included bias, accountability, governance, and explainability. Recommendations identified in the literature to address present barriers call for a proactive evaluation of MMS, multidisciplinary collaboration, adherence to investigation and validation protocols, transparency and traceability requirements, and guiding standards and frameworks that enhance documentation efforts and support the translation of AI- and ML-based MMS. Conclusions: Resolving barriers to translation is critical for MMS to deliver on expectations, including those barriers identified in this scoping review related to bias, accountability, governance, and explainability. Our findings suggest that transparent strategic documentation, aligning translational science and regulatory science, will support the translation of MMS by coordinating communication and reporting and reducing translational barriers, thereby furthering the adoption of MMS.
- 14 July 2023
- Published by JMIR Publications Inc. in Interactive Journal of Medical Research
- Vol. 12 (1) , e45903
Open AccessThe Role of Artificial Intelligence Model Documentation in Translational Science: Scoping Review
Background: Despite the touted potential of artificial intelligence (AI) and machine learning (ML) to revolutionize health care, clinical decision support tools, herein referred to as medical modeling software (MMS), have yet to realize the anticipated benefits. One proposed obstacle is the acknowledged gaps in AI translation. These gaps stem partly from the fragmentation of processes and resources to support MMS transparent documentation. Consequently, the absence of transparent reporting hinders the provision of evidence to support the implementation of MMS in clinical practice, thereby serving as a substantial barrier to the successful translation of software from research settings to clinical practice. Objective: This study aimed to scope the current landscape of AI- and ML-based MMS documentation practices and elucidate the function of documentation in facilitating the translation of ethical and explainable MMS into clinical workflows. Methods: A scoping review was conducted in accordance with PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines. PubMed was searched using Medical Subject Headings key concepts of AI, ML, ethical considerations, and explainability to identify publications detailing AI- and ML-based MMS documentation, in addition to snowball sampling of selected reference lists. To include the possibility of implicit documentation practices not explicitly labeled as such, we did not use documentation as a key concept but as an inclusion criterion. A 2-stage screening process (title and abstract screening and full-text review) was conducted by 1 author. A data extraction template was used to record publication-related information; barriers to developing ethical and explainable MMS; available standards, regulations, frameworks, or governance strategies related to documentation; and recommendations for documentation for papers that met the inclusion criteria. Results: Of the 115 papers retrieved, 21 (18.3%) papers met the requirements for inclusion. Ethics and explainability were investigated in the context of AI- and ML-based MMS documentation and translation. Data detailing the current state and challenges and recommendations for future studies were synthesized. Notable themes defining the current state and challenges that required thorough review included bias, accountability, governance, and explainability. Recommendations identified in the literature to address present barriers call for a proactive evaluation of MMS, multidisciplinary collaboration, adherence to investigation and validation protocols, transparency and traceability requirements, and guiding standards and frameworks that enhance documentation efforts and support the translation of AI- and ML-based MMS. Conclusions: Resolving barriers to translation is critical for MMS to deliver on expectations, including those barriers identified in this scoping review related to bias, accountability, governance, and explainability. Our findings suggest that transparent strategic documentation, aligning translational science and regulatory science, will support the translation of MMS by coordinating communication and reporting and reducing translational barriers, thereby furthering the adoption of MMS.
- Open Access
- 25 January 2023
- Published by JMIR Publications Inc. in JMIR Preprints
The Role of Artificial Intelligence Model Documentation in Translational Science: Scoping Review (Preprint)
BACKGROUND Despite the touted potential of artificial intelligence (AI) and machine learning (ML) to revolutionize health care, clinical decision support tools, herein referred to as medical modeling software (MMS), have yet to realize the anticipated benefits. One proposed obstacle is the acknowledged gaps in AI translation. These gaps stem partly from the fragmentation of processes and resources to support MMS transparent documentation. Consequently, the absence of transparent reporting hinders the provision of evidence to support the implementation of MMS in clinical practice, thereby serving as a substantial barrier to the successful translation of software from research settings to clinical practice. OBJECTIVE This study aimed to scope the current landscape of AI- and ML-based MMS documentation practices and elucidate the function of documentation in facilitating the translation of ethical and explainable MMS into clinical workflows. METHODS A scoping review was conducted in accordance with PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines. PubMed was searched using Medical Subject Headings key concepts of AI, ML, ethical considerations, and explainability to identify publications detailing AI- and ML-based MMS documentation, in addition to snowball sampling of selected reference lists. To include the possibility of implicit documentation practices not explicitly labeled as such, we did not use documentation as a key concept but as an inclusion criterion. A 2-stage screening process (title and abstract screening and full-text review) was conducted by 1 author. A data extraction template was used to record publication-related information; barriers to developing ethical and explainable MMS; available standards, regulations, frameworks, or governance strategies related to documentation; and recommendations for documentation for papers that met the inclusion criteria. RESULTS Of the 115 papers retrieved, 21 (18.3%) papers met the requirements for inclusion. Ethics and explainability were investigated in the context of AI- and ML-based MMS documentation and translation. Data detailing the current state and challenges and recommendations for future studies were synthesized. Notable themes defining the current state and challenges that required thorough review included bias, accountability, governance, and explainability. Recommendations identified in the literature to address present barriers call for a proactive evaluation of MMS, multidisciplinary collaboration, adherence to investigation and validation protocols, transparency and traceability requirements, and guiding standards and frameworks that enhance documentation efforts and support the translation of AI- and ML-based MMS. CONCLUSIONS Resolving barriers to translation is critical for MMS to deliver on expectations, including those barriers identified in this scoping review related to bias, accountability, governance, and explainability. Our findings suggest that transparent strategic documentation, aligning translational science and regulatory science, will support the translation of MMS by coordinating communication and reporting and reducing translational barriers, thereby furthering the adoption of MMS.
- 25 January 2023
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessThe Role of Artificial Intelligence Model Documentation in Translational Science: Scoping Review (Preprint)
BACKGROUND Despite the touted potential of artificial intelligence (AI) and machine learning (ML) to revolutionize health care, clinical decision support tools, herein referred to as medical modeling software (MMS), have yet to realize the anticipated benefits. One proposed obstacle is the acknowledged gaps in AI translation. These gaps stem partly from the fragmentation of processes and resources to support MMS transparent documentation. Consequently, the absence of transparent reporting hinders the provision of evidence to support the implementation of MMS in clinical practice, thereby serving as a substantial barrier to the successful translation of software from research settings to clinical practice. OBJECTIVE This study aimed to scope the current landscape of AI- and ML-based MMS documentation practices and elucidate the function of documentation in facilitating the translation of ethical and explainable MMS into clinical workflows. METHODS A scoping review was conducted in accordance with PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines. PubMed was searched using Medical Subject Headings key concepts of AI, ML, ethical considerations, and explainability to identify publications detailing AI- and ML-based MMS documentation, in addition to snowball sampling of selected reference lists. To include the possibility of implicit documentation practices not explicitly labeled as such, we did not use documentation as a key concept but as an inclusion criterion. A 2-stage screening process (title and abstract screening and full-text review) was conducted by 1 author. A data extraction template was used to record publication-related information; barriers to developing ethical and explainable MMS; available standards, regulations, frameworks, or governance strategies related to documentation; and recommendations for documentation for papers that met the inclusion criteria. RESULTS Of the 115 papers retrieved, 21 (18.3%) papers met the requirements for inclusion. Ethics and explainability were investigated in the context of AI- and ML-based MMS documentation and translation. Data detailing the current state and challenges and recommendations for future studies were synthesized. Notable themes defining the current state and challenges that required thorough review included bias, accountability, governance, and explainability. Recommendations identified in the literature to address present barriers call for a proactive evaluation of MMS, multidisciplinary collaboration, adherence to investigation and validation protocols, transparency and traceability requirements, and guiding standards and frameworks that enhance documentation efforts and support the translation of AI- and ML-based MMS. CONCLUSIONS Resolving barriers to translation is critical for MMS to deliver on expectations, including those barriers identified in this scoping review related to bias, accountability, governance, and explainability. Our findings suggest that transparent strategic documentation, aligning translational science and regulatory science, will support the translation of MMS by coordinating communication and reporting and reducing translational barriers, thereby furthering the adoption of MMS.
- Open Access
- 23 January 2023
- Published by JMIR Publications Inc. in JMIR Preprints
Clinical Needs Assessment of a Machine Learning–Based Asthma Management Tool: User-Centered Design Approach (Preprint)
BACKGROUND Personalized asthma management depends on a clinician’s ability to efficiently review patient’s data and make timely clinical decisions. Unfortunately, efficient and effective review of these data is impeded by the varied format, location, and workflow of data acquisition, storage, and processing in the electronic health record. While machine learning (ML) and clinical decision support tools are well-positioned as potential solutions, the translation of such frameworks requires that barriers to implementation be addressed in the formative research stages. OBJECTIVE We aimed to use a structured user-centered design approach (double-diamond design framework) to (1) qualitatively explore clinicians’ experience with the current asthma management system, (2) identify user requirements to improve algorithm explainability and Asthma Guidance and Prediction System prototype, and (3) identify potential barriers to ML-based clinical decision support system use. METHODS At the “discovery” phase, we first shadowed to understand the practice context. Then, semistructured interviews were conducted digitally with 14 clinicians who encountered pediatric asthma patients at 2 outpatient facilities. Participants were asked about their current difficulties in gathering information for patients with pediatric asthma, their expectations of ideal workflows and tools, and suggestions on user-centered interfaces and features. At the “define” phase, a synthesis analysis was conducted to converge key results from interviewees’ insights into themes, eventually forming critical “how might we” research questions to guide model development and implementation. RESULTS We identified user requirements and potential barriers associated with three overarching themes: (1) usability and workflow aspects of the ML system, (2) user expectations and algorithm explainability, and (3) barriers to implementation in context. Even though the responsibilities and workflows vary among different roles, the core asthma-related information and functions they requested were highly cohesive, which allows for a shared information view of the tool. Clinicians hope to perceive the usability of the model with the ability to note patients’ high risks and take proactive actions to manage asthma efficiently and effectively. For optimal ML algorithm explainability, requirements included documentation to support the validity of algorithm development and output logic, and a request for increased transparency to build trust and validate how the algorithm arrived at the decision. Acceptability, adoption, and sustainability of the asthma management tool are implementation outcomes that are reliant on the proper design and training as suggested by participants. CONCLUSIONS As part of our comprehensive informatics-based process centered on clinical usability, we approach the problem using a theoretical framework grounded in user experience research leveraging semistructured interviews. Our focus on meeting the needs of the practice with ML technology is emphasized by a user-centered approach to clinician engagement through upstream technology design.
- 23 January 2023
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessClinical Needs Assessment of a Machine Learning–Based Asthma Management Tool: User-Centered Design Approach (Preprint)
BACKGROUND Personalized asthma management depends on a clinician’s ability to efficiently review patient’s data and make timely clinical decisions. Unfortunately, efficient and effective review of these data is impeded by the varied format, location, and workflow of data acquisition, storage, and processing in the electronic health record. While machine learning (ML) and clinical decision support tools are well-positioned as potential solutions, the translation of such frameworks requires that barriers to implementation be addressed in the formative research stages. OBJECTIVE We aimed to use a structured user-centered design approach (double-diamond design framework) to (1) qualitatively explore clinicians’ experience with the current asthma management system, (2) identify user requirements to improve algorithm explainability and Asthma Guidance and Prediction System prototype, and (3) identify potential barriers to ML-based clinical decision support system use. METHODS At the “discovery” phase, we first shadowed to understand the practice context. Then, semistructured interviews were conducted digitally with 14 clinicians who encountered pediatric asthma patients at 2 outpatient facilities. Participants were asked about their current difficulties in gathering information for patients with pediatric asthma, their expectations of ideal workflows and tools, and suggestions on user-centered interfaces and features. At the “define” phase, a synthesis analysis was conducted to converge key results from interviewees’ insights into themes, eventually forming critical “how might we” research questions to guide model development and implementation. RESULTS We identified user requirements and potential barriers associated with three overarching themes: (1) usability and workflow aspects of the ML system, (2) user expectations and algorithm explainability, and (3) barriers to implementation in context. Even though the responsibilities and workflows vary among different roles, the core asthma-related information and functions they requested were highly cohesive, which allows for a shared information view of the tool. Clinicians hope to perceive the usability of the model with the ability to note patients’ high risks and take proactive actions to manage asthma efficiently and effectively. For optimal ML algorithm explainability, requirements included documentation to support the validity of algorithm development and output logic, and a request for increased transparency to build trust and validate how the algorithm arrived at the decision. Acceptability, adoption, and sustainability of the asthma management tool are implementation outcomes that are reliant on the proper design and training as suggested by participants. CONCLUSIONS As part of our comprehensive informatics-based process centered on clinical usability, we approach the problem using a theoretical framework grounded in user experience research leveraging semistructured interviews. Our focus on meeting the needs of the practice with ML technology is emphasized by a user-centered approach to clinician engagement through upstream technology design.
- Open Access
The Role of AI Model Documentation in Translational Science: A Scoping Review
Background Translation of artificial intelligence/machine learning (AI/ML)-based medical modeling software (MMS) into clinical settings requires rigorous evaluation by interdisciplinary teams and across the AI lifecycle. The fragmented nature of available resources to support MMS documentation limits the transparent reporting of scientific evidence to support MMS, creating barriers and impeding the translation of software from code to bedside. Objective The aim of this paper is to scope AI/ML-based MMS documentation practices and define the role of documentation in facilitating safe and ethical MMS translation into clinical workflows. Methods A scoping review was conducted in accordance with PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. MEDLINE (PubMed) was searched using MeSH key concepts of AI/ML, ethical considerations, and explainability to identify publications detailing AI/ML-based MMS documentation, in addition to snowball sampling of selected reference lists. To include the possibility of implicit documentation practices not explicitly labeled as such, we did not use “documentation “ as a key concept but rather as an inclusion criterion. A two-stage screening process (title and abstract screening and full-text review) was conducted by an independent reviewer. A data extraction template was utilized to record publication-related information, barriers to developing ethical and explainable MMS, available standards, regulations, frameworks, or governance strategies related to documentation, and recommendations for documentation for papers that met inclusion criteria. Results Of the total 115 papers, 21 (18%) articles met the requirements for inclusion. Data regarding the current state and challenges of AI/ML-based documentation was synthesized and themes including bias, accountability, governance, and interpretability were identified. Conclusions Our findings suggest that AI/ML-based MMS documentation practice is siloed across the AI life cycle and there exists a gray area for tracking and reporting of non-regulated MMS. Recommendations from the literature call for proactive evaluation, standards, frameworks, and transparency and traceability requirements to address ethical and explainability barriers, enhance documentation efforts, provide support throughout the AI lifecycle, and promote translation of MMS. If prioritized across multidisciplinary teams and across the AI lifecycle, AI/ML-based MMS documentation may serve as a method of coordinated communication and reporting toward resolution of AI translation barriers related to bias, accountability, governance, and interpretability.
Open AccessThe Role of AI Model Documentation in Translational Science: A Scoping Review
Background Translation of artificial intelligence/machine learning (AI/ML)-based medical modeling software (MMS) into clinical settings requires rigorous evaluation by interdisciplinary teams and across the AI lifecycle. The fragmented nature of available resources to support MMS documentation limits the transparent reporting of scientific evidence to support MMS, creating barriers and impeding the translation of software from code to bedside. Objective The aim of this paper is to scope AI/ML-based MMS documentation practices and define the role of documentation in facilitating safe and ethical MMS translation into clinical workflows. Methods A scoping review was conducted in accordance with PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. MEDLINE (PubMed) was searched using MeSH key concepts of AI/ML, ethical considerations, and explainability to identify publications detailing AI/ML-based MMS documentation, in addition to snowball sampling of selected reference lists. To include the possibility of implicit documentation practices not explicitly labeled as such, we did not use “documentation “ as a key concept but rather as an inclusion criterion. A two-stage screening process (title and abstract screening and full-text review) was conducted by an independent reviewer. A data extraction template was utilized to record publication-related information, barriers to developing ethical and explainable MMS, available standards, regulations, frameworks, or governance strategies related to documentation, and recommendations for documentation for papers that met inclusion criteria. Results Of the total 115 papers, 21 (18%) articles met the requirements for inclusion. Data regarding the current state and challenges of AI/ML-based documentation was synthesized and themes including bias, accountability, governance, and interpretability were identified. Conclusions Our findings suggest that AI/ML-based MMS documentation practice is siloed across the AI life cycle and there exists a gray area for tracking and reporting of non-regulated MMS. Recommendations from the literature call for proactive evaluation, standards, frameworks, and transparency and traceability requirements to address ethical and explainability barriers, enhance documentation efforts, provide support throughout the AI lifecycle, and promote translation of MMS. If prioritized across multidisciplinary teams and across the AI lifecycle, AI/ML-based MMS documentation may serve as a method of coordinated communication and reporting toward resolution of AI translation barriers related to bias, accountability, governance, and interpretability.