Enhancing Model Generalizability in Medical Artificial Intelligence: Systematic Comparison of Categorical Encoding and Sampling Techniques for Imbalanced Data
Open Access
- 13 April 2026
- journal article
- research article
- Published by JMIR Publications Inc. in JMIR Medical Informatics
- Vol. 14, e75655
Abstract
Background: Despite the increasing use of machine learning (ML) in clinical research, the early stages of data preparation, especially for structured clinical data, often receive limited methodological scrutiny. These datasets typically contain missing values, complex categorical variables, and imbalanced class distributions, all of which complicate downstream model development and interpretation. Objective: This study introduces a structured preprocessing framework designed to address common challenges in medical tabular data and to assess how preprocessing choices affect the stability and portability of predictive models across settings. Methods: We constructed a modular workflow comprising 3 components. First, preprocessing strategies included imputation for missing data, 3 types of categorical encoding (one-hot, frequency, and target), and resampling approaches for class imbalance (Synthetic Minority Over-sampling Technique [SMOTE] and Random Over Sampling Example [ROSE]). Second, 6 classification algorithms were used to evaluate performance patterns, including logistic regression (LGR), decision tree (DT), random forest, XGBoost (XGB), CatBoost (CAT), and light gradient-boosting machine (LightGBM). Third, we assessed cross-dataset portability using 2 datasets with distinct data-generating mechanisms: a registry for patients with end-stage renal disease (ESRD; n=412) and the population-based Behavioral Risk Factor Surveillance System (BRFSS) 2015 survey. For each dataset, we independently cleaned, standardized, encoded, tuned, and evaluated models using the same predefined hyperparameter search space, without cross-dataset feature matching or pooling the area under the ROC curve (AUC) calculations; the complete pipeline was then rerun on BRFSS as an external replication. Results: One-hot encoding in combination with ROSE yielded the most consistent performance improvements in terms of AUC (0.940) and accuracy (0.932), particularly for classifiers sensitive to class distribution. Notably, ROSE enhanced sensitivity without substantially distorting the original data structure. Feature importance rankings also contributed to model interpretability, and performance trends were largely reproducible in cross-context application. Conclusions: Our findings suggest that preprocessing decisions often treated as ancillary play a central role in shaping model outcomes, especially in high-variance clinical datasets. The proposed framework offers a reproducible and adaptable tool for aligning data preparation with the unique demands of health care prediction tasks and may serve as a foundation for future efforts to standardize preprocessing in clinical ML workflows.Keywords
This publication has 40 references indexed in Scilit:
- Combinations of valvular calcification and serum alkaline phosphatase predict cardiovascular risk among end-stage kidney disease patientsIJC Heart & Vasculature, 2024
- Diabetes mellitus prediction and diagnosis from a data preprocessing and machine learning perspectiveComputer Methods and Programs in Biomedicine, 2022
- Regularized target encoding outperforms traditional methods in supervised machine learning with high cardinality featuresComputational Statistics, 2022
- Deep learning for temporal data representation in electronic health records: A systematic review of challenges and methodologiesJournal of Biomedical Informatics, 2022
- Effects of Resampling Techniques on Imbalanced Data Classification: A New Under-resampling Method2021
- Systematic Review of Approaches to Preserve Machine Learning Performance in the Presence of Temporal Dataset Shift in Clinical MedicineApplied Clinical Informatics, 2021
- Missing Data Preprocessing in Credit Classification: One-Hot Encoding or Imputation?Emerging Markets Finance and Trade, 2020
- Cross-Validation2019
- Training and assessing classification rules with imbalanced dataData Mining and Knowledge Discovery, 2012
- MissForest—non-parametric missing value imputation for mixed-type dataBioinformatics, 2011