Improving the Reliability of Medical Diagnostic Models through Rule-Based Decision Deferral
Open Access
- 3 October 2023
- journal article
- research article
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Symposium Series
- Vol. 1 (1) , 122-126
Abstract
Pancreatic ductal adenocarcinoma (PDAC) is a highly lethal cancer, and accurate assessment of tumor resectability is crucial for determining appropriate treatment. AI-based models have shown promise in classifying tumor resectability, but reliability concerns have impeded clinical implementation. We propose extending the AI-based VasQNet model for classifying tumor resectability on AI-generated segmentations of computed tomography scans (CTs) to improve the models’ reliability. This extension allows VasQNet to defer decisions when the AI-generated segmentations violate pre-established rules on vascular anatomy, tumor location, and tumor size. We conducted experiments using CTs of (borderline) resectable and non-resectable PDAC patients. We evaluated the performance of the baseline VasQNet and the extended VasQNet with rule-based decision deferral (RBDD) by comparing their classifications to a ground-truth provided by a radiologist, employing agreement as a metric. Our results demonstrate that the extended VasQNet achieved a significantly higher agreement (90%) with the radiologist’s classification than the baseline VasQNet (67%). Notably, 17/31 (54%) deferred decisions would have been incorrect had they not been deferred. Our study demonstrates the effectiveness of RBDD in improving the reliability of clinical diagnostic models through the exemplification of VasQNet. In conclusion, RBDD can enhance the reliability of clinical diagnostics models, facilitating integration into clinical practice. The documented code is available on GitHub (https://github.com/PHAIR-Consortium/Vessel-Involvement-Quantifier).Keywords
This publication has 13 references indexed in Scilit:
- Bridging the chasm between AI and clinical implementationThe Lancet, 2022
- Incorporating uncertainty in learning to defer algorithms for safe computer-aided diagnosisScientific Reports, 2022
- A review of uncertainty quantification in deep learning: Techniques, applications and challengesInformation Fusion, 2021
- Uncertainty quantification in skin cancer classification using three-way decision-based Bayesian deep learningComputers in Biology and Medicine, 2021
- CT-diagnosed extra-pancreatic extension of pancreatic ductal adenocarcinoma is a more reliable prognostic factor for survival than pathology-diagnosed extensionEuropean Radiology, 2021
- Modern Monte Carlo methods for efficient uncertainty quantification and propagation: A surveyWIREs Computational Statistics, 2020
- Uncertainty quantification in epidemiological models for the COVID-19 pandemicComputers in Biology and Medicine, 2020
- Quantifying vascular invasion in pancreatic cancer—a contrast CT based method for surgical resectability evaluationPhysics in Medicine & Biology, 2020
- Performance of CT-based radiomics in diagnosis of superior mesenteric vein resection margin in patients with pancreatic head cancerAbdominal Radiology, 2020
- Current Approaches to Pancreatic Cancer ScreeningThe American Journal of Pathology, 2019