Leakage-Aware Evaluation of Ensemble Classifiers for Symptom-Based Classification of Diabetes Status: Performance, Calibration, and Explainability
Authors:
- Daniel Andrade-GirónDADaniel Andrade-Girón
- William Marin-RodriguezWilliam Marin-Rodriguez
- Americo PeñaAPAmerico Peña
- Enrique Diaz-VegaEDEnrique Diaz-Vega
- Abrahán Neri-AyalaANAbrahán Neri-Ayala
- Viviana Vellon-FloresVVViviana Vellon-Flores
- Timoteo Solano-ArmasTSTimoteo Solano-Armas
- Algemiro Muñoz-VilelaAMAlgemiro Muñoz-Vilela
Preprint
- 20 September 2026
- preprint
- Published by MDPI AG in Preprints.org
Abstract
Background/Objectives: Diabetes impose a substantial public health burden. Machine learning models trained on symptom data may help identify individuals whose recorded symptom profiles warrant confirmatory clinical assessment. This study compared ten en-semble models for symptom-based classification of a dataset-provided diabetes-status la-bel. Methods: The UCI Early-Stage Diabetes Risk Prediction Dataset was used, comprising 520 observations, 251 unique predictor profiles, 16 predictors, and a binary positive/negative diabetes-status label. All observations were retained, while identical predictor profiles were kept within the same partition during data splitting, nested cross-validation, hy-perparameter optimization, and calibration to reduce information leakage. The weighted F1-score was the optimization objective and primary model-selection metric throughout the analysis. Results: Bagging Extra Tree achieved the highest observed mean weighted F1-score (94.07% ± 7.99%) and MCC (0.876 ± 0.167) and was selected as the final model under the prespecified criterion. An exploratory Friedman analysis indicated variation in model rankings across folds (p = 0.001859), although no Nemenyi-adjusted pairwise comparison reached the nominal significance level. On the profile-disjoint internal holdout set, the calibrated model achieved a weighted F1-score of 0.9224, an MCC of 0.8207, a ROC-AUC of 0.9844, and a Brier score of 0.0528. Random Forest was selected in the unique-profile sensitivity analysis but yielded lower holdout performance, indicating sensitivity to the analytical unit. Conclusions: These findings provide internal evidence that tree-based ensembles can clas-sify the dataset-provided diabetes-status label under profile-disjoint evaluation. They do not establish prospective risk prediction or clinical diagnostic utility; independent exter-nal validation is required before clinical implementation.Keywords
All Related Versions
This publication has 80 references indexed in Scilit:
- Interpretable Machine Learning: A Comprehensive Review of Foundations, Methods, and the Path ForwardWIREs Data Mining and Knowledge Discovery, 2026
- Multicriteria Evaluation of Supervised Classification Algorithms: Strengths, Limitations and Practical Recommendations2025
- Pairwise statistical comparisons of multiple algorithmsFrontiers of Computer Science, 2025
- Detecting diabetes in an ensemble model using a unique PSO-GWO hybrid approach to hyperparameter optimizationNeural Computing and Applications, 2024
- Intrusion detection based on ensemble learning for big data classificationCluster Computing, 2023
- Genetic Algorithm and Naïve Bayes-Based (GANB) Diabetes Mellitus Prediction System2022
- Prediction model using SMOTE, genetic algorithm and decision tree (PMSGD) for classification of diabetes mellitusMultimedia Systems, 2021
- Automatic Diabetes and Liver Disease Diagnosis and Prediction Through SVM and KNN Algorithms2021
- Use of a K-nearest neighbors model to predict the development of type 2 diabetes within 2 years in an obese, hypertensive populationMedical & Biological Engineering & Computing, 2020
- Generating ensembles of heterogeneous classifiers using Stacked GeneralizationWIREs Data Mining and Knowledge Discovery, 2015