Assessing Barriers to Implementation of Machine Learning and Artificial Intelligence–Based Tools in Critical Care: Web-Based Survey Study
Open Access
- 27 January 2023
- journal article
- research article
- Published by JMIR Publications Inc. in JMIR Perioperative Medicine
- Vol. 6 (1) , e41056
Abstract
Background: Although there is considerable interest in machine learning (ML) and artificial intelligence (AI) in critical care, the implementation of effective algorithms into practice has been limited. Objective: We sought to understand physician perspectives of a novel intubation prediction tool. Further, we sought to understand health care provider and nonprovider perspectives on the use of ML in health care. We aim to use the data gathered to elucidate implementation barriers and determinants of this intubation prediction tool, as well as ML/AI-based algorithms in critical care and health care in general. Methods: We developed 2 anonymous surveys in Qualtrics, 1 single-center survey distributed to 99 critical care physicians via email, and 1 social media survey distributed via Facebook and Twitter with branching logic to tailor questions for providers and nonproviders. The surveys included a mixture of categorical, Likert scale, and free-text items. Likert scale means with SD were reported from 1 to 5. We used student t tests to examine the differences between groups. In addition, Likert scale responses were converted into 3 categories, and percentage values were reported in order to demonstrate the distribution of responses. Qualitative free-text responses were reviewed by a member of the study team to determine validity, and content analysis was performed to determine common themes in responses. Results: Out of 99 critical care physicians, 47 (48%) completed the single-center survey. Perceived knowledge of ML was low with a mean Likert score of 2.4 out of 5 (SD 0.96), with 7.5% of respondents rating their knowledge as a 4 or 5. The willingness to use the ML-based algorithm was 3.32 out of 5 (SD 0.95), with 75% of respondents answering 3 out of 5. The social media survey had 770 total responses with 605 (79%) providers and 165 (21%) nonproviders. We found no difference in providers’ perceived knowledge based on level of experience in either survey. We found that nonproviders had significantly less perceived knowledge of ML (mean 3.04 out of 5, SD 1.53 vs mean 3.43, SD 0.941; P<.001) and comfort with ML (mean 3.28 out of 5, SD 1.02 vs mean 3.53, SD 0.935; P=.004) than providers. Free-text responses revealed multiple shared concerns, including accuracy/reliability, data bias, patient safety, and privacy/security risks. Conclusions: These data suggest that providers and nonproviders have positive perceptions of ML-based tools, and that a tool to predict the need for intubation would be of interest to critical care providers. There were many shared concerns about ML/AI in health care elucidated by the surveys. These results provide a baseline evaluation of implementation barriers and determinants of ML/AI-based tools that will be important in their optimal implementation and adoption in the critical care setting and health care in general.Keywords
All Related Versions
This publication has 24 references indexed in Scilit:
- An Index Combining Respiratory Rate and Oxygenation to Predict Outcome of Nasal High-Flow TherapyAmerican Journal of Respiratory and Critical Care Medicine, 2019
- Artificial Intelligence and the Future of Primary Care: Exploratory Qualitative Study of UK General Practitioners’ ViewsJournal of Medical Internet Research, 2019
- Automatic Pulmonary Nodule Detection Applying Deep Learning or Machine Learning Algorithms to the LIDC-IDRI Database: A Systematic ReviewDiagnostics, 2019
- High-performance medicine: the convergence of human and artificial intelligenceNature Medicine, 2019
- A Twitter-based survey on marijuana concentrate useDrug and Alcohol Dependence, 2018
- An Interpretable Machine Learning Model for Accurate Prediction of Sepsis in the ICUCritical Care Medicine, 2018
- Delirium and sedation in the intensive care unit: Survey of behaviors and attitudes of 1384 healthcare professionals*Critical Care Medicine, 2009
- Barriers to low tidal volume ventilation in acute respiratory distress syndrome: Survey development, validation, and resultsCritical Care Medicine, 2007
- Cognitive Processing of Self-Report Items in Educational Research: Do They Think What We Mean?Educational Psychologist, 2007
- Improving the Quality of Web Surveys: The Checklist for Reporting Results of Internet E-Surveys (CHERRIES)Journal of Medical Internet Research, 2004