TRIAGE: Trustworthy Reporting and Assessment for Clinical Gain and Effectiveness of AI Models
Authors:
- Farzaneh FazilatiFFFarzaneh Fazilati
- Mohammad Zakaria RajabiMRMohammad Zakaria Rajabi
- Nima AlihosseiniNANima Alihosseini
- Mohaddeseh Esmaeili FarsaniMFMohaddeseh Esmaeili Farsani
- Seyed Hasan SandidSSSeyed Hasan Sandid
- Shadi ZamaniSZShadi Zamani
- Mehrshad Alirezaei FarahaniMFMehrshad Alirezaei Farahani
- Fateme BiriaeiFBFateme Biriaei
- Fateme SadeghipourFSFateme Sadeghipour
- Mohammad Taha MirshamsiMohammad Taha Mirshamsi
- Mottahareh FahamiMFMottahareh Fahami
- Hamid Reza MaratebHamid Reza Marateb
Open Access
- 25 February 2026
- journal article
- review article
- Published by MDPI AG in Diagnostics
- Vol. 16 (5) , 666
Abstract
Machine learning (ML), including deep learning, kernel-based classifiers, and ensemble methods, is increasingly used to support clinical diagnosis in medical imaging, biosignal interpretation, and electronic health record (EHR)-based decision support. Despite rapid progress, many diagnostic AI studies still rely on limited retrospective evaluation and single summary measures (e.g., accuracy or AUC), creating a gap between reported model performance and evidence required for safe clinical adoption. This review proposes TRIAGE, a clinically grounded evaluation framework designed to organize diagnostic AI testing as an evidence pipeline aligned with real clinical use cases (screening, triage, second reading, and confirmatory testing). We summarize core discrimination metrics derived from the confusion matrix (sensitivity, specificity, predictive values, likelihood ratios, diagnostic odds ratio, and F-scores) and highlight the importance of prevalence and spectrum effects for interpreting predictive value and clinical workload. We further review evaluation strategies for multi-class and multi-label diagnostic tasks using appropriate aggregation methods (micro, macro, and weighted averaging) and set-based measures such as Hamming loss, exact match ratio, and Jaccard/IoU. Because diagnostic deployment is threshold-dependent, we integrate representation curves (ROC, precision–recall, lift, and cumulative gain) with calibration assessment and clinical utility analysis, including calibration slope, Brier score, and decision-curve analysis. We also address robustness and fairness evaluation, leakage-resistant validation designs (patient-grouped splits, stratified and temporal validation, and external validation), computational constraints relevant to deployment (latency, throughput, and energy use), and statistically sound model comparison with multiplicity control. A structured TRIAGE checklist table summarizing the evaluation parameters described in this review is provided in the main text to support reproducible and clinically interpretable reporting.Keywords
This publication has 125 references indexed in Scilit:
- Detecting and Remediating Harmful Data Shifts for the Responsible Deployment of Clinical AI ModelsJAMA Network Open, 2025
- Automation Bias and Assistive AIJAMA, 2023
- Effective use of the McNemar testBehavioral Ecology and Sociobiology, 2020
- Paired Samples T-Test2017
- Recommended tests and confidence intervals for paired binomial proportionsStatistics in Medicine, 2014
- Stratified Fisher's exact test and its sample size calculationBiometrical Journal, 2013
- A History of the Central Limit Theorem2011
- Kruskal‐Wallis Test2010
- Chi‐squared and Fisher–Irwin tests of two‐by‐two tables with small sample recommendationsStatistics in Medicine, 2007
- Central Limit Theorems for dependent variables. IProbability Theory and Related Fields, 1981