Entropy Guided Benchmarking of Classical and Generative Imputation Methods for High-Dimensional Healthcare Survey Data

Authors:
Open Access
Publisher website
Google Scholar
Abstract
Missing data are a persistent challenge in large healthcare datasets, often undermining both statistical validity and machine learning performance when handled using simplistic assumptions. In this work, we examine how entropy-based diagnostics can guide the selection of imputation strategies for high-dimensional health survey data using the National Health and Nutrition Examination Survey (NHANES) 2021–2023. Shannon entropy is used to identify variables with structurally complex missingness, and a range of classical approaches (mean imputation, k-nearest neighbors, and multiple imputation by chained equations) are evaluated alongside deep generative methods, including variational autoencoders, generative adversarial networks (GANs), Wasserstein GANs (WGANs), and diffusion-based models. All methods are compared under a controlled masked-entry evaluation using root mean square error (RMSE) and Kolmogorov–Smirnov (KS) statistics to capture both reconstruction accuracy and distributional fidelity. Results show that diffusion-based models provide the most consistent balance between numerical accuracy and distributional preservation across high-entropy dietary variables, while WGAN demonstrates competitive performance for selected distributions. Structural equation modeling further indicates that entropy is a useful diagnostic signal for identifying variables that are difficult to reconstruct. Overall, this study provides a reproducible framework for aligning imputation strategy with missingness complexity in healthcare data, with implications for improving reliability in downstream analytics.