Resilient Artificial Intelligence in Health: Synthesis and Research Agenda Toward Next-Generation Trustworthy Clinical Decision Support
Open Access
- 28 June 2024
- journal article
- research article
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 26 (1) , e50295
Abstract
Artificial intelligence (AI)–based clinical decision support systems are gaining momentum by relying on a greater volume and variety of secondary use data. However, the uncertainty, variability, and biases in real-world data environments still pose significant challenges to the development of health AI, its routine clinical use, and its regulatory frameworks. Health AI should be resilient against real-world environments throughout its lifecycle, including the training and prediction phases and maintenance during production, and health AI regulations should evolve accordingly. Data quality issues, variability over time or across sites, information uncertainty, human-computer interaction, and fundamental rights assurance are among the most relevant challenges. If health AI is not designed resiliently with regard to these real-world data effects, potentially biased data-driven medical decisions can risk the safety and fundamental rights of millions of people. In this viewpoint, we review the challenges, requirements, and methods for resilient AI in health and provide a research framework to improve the trustworthiness of next-generation AI-based clinical decision support.Keywords
All Related Versions
This publication has 47 references indexed in Scilit:
- Extremely missing numerical data in Electronic Health Records for machine learning can be managed through simple imputation methods considering informative missingness: A comparative of solutions in a COVID-19 mortality case studyComputer Methods and Programs in Biomedicine, 2023
- A survey of uncertainty in deep neural networksArtificial Intelligence Review, 2023
- Attention is not all you need: the complicated case of ethically using large language models in healthcare and medicineEBioMedicine, 2023
- Operationalising ethics in artificial intelligence for healthcare: a framework for AI developersAI and Ethics, 2022
- Legal aspects of data cleansing in medical AIComputer Law & Security Review, 2021
- HUMAN‐CENTERED DESIGN OF ARTIFICIAL INTELLIGENCE2021
- Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methodsMachine Learning, 2021
- Potential Biases in Machine Learning Algorithms Using Electronic Health Record DataJAMA Internal Medicine, 2018
- Clinical Implications and Challenges of Artificial Intelligence and Deep LearningJAMA, 2018
- Unintended Consequences of Machine Learning in MedicineJAMA, 2017