A Combined Interpolation and Weighted K-Nearest Neighbours Approach for the Imputation of Longitudinal ICU Laboratory Data
- 2 March 2020
- journal article
- research article
- Published by Springer Nature in Journal of Healthcare Informatics Research
- Vol. 4 (2) , 174-188
Abstract
The presence of missing data is a common problem that affects almost all clinical datasets. Since most available data mining and machine learning algorithms require complete datasets, accurately imputing (i.e. “filling in”) the missing data is an essential step. This paper presents a methodology for the missing data imputation of longitudinal clinical data based on the integration of linear interpolation and a weighted K-Nearest Neighbours (KNN) algorithm. The Maximal Information Coefficient (MIC) values among features are employed as weights for the distance computation in the KNN algorithm in order to integrate intra- and inter-patient information. An interpolation-based imputation approach was also employed and tested both independently and in combination with the KNN algorithm. The final imputation is carried out by applying the best performing method for each feature. The methodology was validated on a dataset of clinical laboratory test results of 13 commonly measured analytes of patients in an intensive care unit (ICU) setting. The performance results are compared with those of 3D-MICE, a state-of-the-art imputation method for cross-sectional and longitudinal patient data. This work was presented in the context of the 2019 ICHI Data Analytics Challenge on Missing data Imputation (DACMI).Keywords
Funding Information
- Italian Ministry of Education, University and Research (”Departments of Excellence” initiative (Law 232/2016))
- Università degli Studi di Padova (C94I19001730001)
This publication has 20 references indexed in Scilit:
- Handling missing data in RCTs; a review of the top medical journalsBMC Medical Research Methodology, 2014
- Comparison of imputation methods for missing laboratory data in medicineBMJ Open, 2013
- Practical and statistical issues in missing data for longitudinal patient-reported outcomesStatistical Methods in Medical Research, 2013
- minerva and minepy: a C engine for the MINE suite and its R, Python and MATLAB wrappersBioinformatics, 2012
- Detecting Novel Associations in Large Data SetsScience, 2011
- MissForest—non-parametric missing value imputation for mixed-type dataBioinformatics, 2011
- Review: A gentle introduction to imputation of missing valuesJournal of Clinical Epidemiology, 2006
- Relationship between haemoglobin and haematocrit in the definition of anaemiaTropical Medicine & International Health, 2006
- Gaussian Processes in Machine Learning2004
- PhysioBank, PhysioToolkit, and PhysioNetCirculation, 2000