Detecting and Mitigating the Clever Hans Effect in Medical Imaging: A Scoping Review
Authors:
- Constanza Vásquez-VenegasCVConstanza Vásquez-Venegas
- Chenwei WuCWChenwei Wu
- Saketh SundarSSSaketh Sundar
- Renata PrôaRPRenata Prôa
- Francis Joshua BeloyFBFrancis Joshua Beloy
- Jillian Reeze MedinaJMJillian Reeze Medina
- Megan McNicholMMMegan McNichol
- Krishnaveni ParvataneniKPKrishnaveni Parvataneni
- Nicholas KurtzmanNKNicholas Kurtzman
- Felipe MirshawkaFMFelipe Mirshawka
- Marcela Aguirre-JerezMAMarcela Aguirre-Jerez
- Daniel K. EbnerDaniel K. Ebner
- 25 November 2024
- journal article
- research article
- Published by Springer Nature in Journal of Imaging Informatics in Medicine
- Vol. 38 (4) , 2563-2579
Abstract
The Clever Hans effect occurs when machine learning models rely on spurious correlations instead of clinically relevant features and poses significant challenges to the development of reliable artificial intelligence (AI) systems in medical imaging. This scoping review provides an overview of methods for identifying and addressing the Clever Hans effect in medical imaging AI algorithms. A total of 173 papers published between 2010 and 2024 were reviewed, and 37 articles were selected for detailed analysis, with classification into two categories: detection and mitigation approaches. Detection methods include model-centric, data-centric, and uncertainty and bias-based approaches, while mitigation strategies encompass data manipulation techniques, feature disentanglement and suppression, and domain knowledge-driven approaches. Despite the progress in detecting and mitigating the Clever Hans effect, the majority of current machine learning studies in medical imaging do not report or test for shortcut learning, highlighting the need for more rigorous validation and transparency in AI research. Future research should focus on creating standardized benchmarks, developing automated detection tools, and exploring the integration of detection and mitigation strategies to comprehensively address shortcut learning. Establishing community-driven best practices and leveraging interdisciplinary collaboration will be crucial for ensuring more reliable, generalizable, and equitable AI systems in healthcare.Keywords
Funding Information
- Foundation for the National Institutes of Health (R01 EB017205, DS-I Africa U54TW012043-01, Bridge2AI OT2OD032701)
- National Science Foundation (ITEST #2148451)
This publication has 55 references indexed in Scilit:
- A Comprehensive Review of Bias in Deep Learning Models: Methods, Impacts, and Future DirectionsArchives of Computational Methods in Engineering, 2024
- Generalisation challenges in deep learning models for medical imagery: insights from external validation of COVID-19 classifiersMultimedia Tools and Applications, 2024
- Beyond Model Accuracy: Identifying Hidden Underlying Issues in Chest X-ray Classification2023
- De-identification and Obfuscation of Gender Attributes from Retinal Scans2023
- Right for the Wrong Reason: Can Interpretable ML Techniques Detect Spurious Correlations?2023
- CFAB: An Online Data Augmentation to Alleviate the Spuriousness of Classification on Medical Ultrasound Images2023
- Removal of Confounders via Invariant Risk Minimization for Medical Diagnosis2022
- Pseudo Bias-Balanced Learning for Debiased Chest X-Ray Classification2022
- A structural characterization of shortcut features for predictionEuropean Journal of Epidemiology, 2022
- Mitigating site effects in covariance for machine learning in neuroimaging dataHuman Brain Mapping, 2021