Impact of Preprocessing Methods on Healthcare Predictions
Preprint
- 9 March 2019
- preprint
- Published by Elsevier in SSRN Electronic Journal
Abstract
Machine learning (ML) is now a day gaining immense importance and is becoming a key technology as the rapid growth of quality of medical data and information. But the early and accurate detection of disease is still a challenge due to the complex, incomplete and multidimensional healthcare data. Data preprocessing is an essential step of ML whose primary goal is to provide processed data to improve the prediction accuracy. This study summarizes the popular data preprocessing steps based on their usages, popularity and literature. After that the selected preprocessing methods is applied on the raw data which is then used by classifiers for predictions. In this experiment we have taken diabetes classification problem. Type II diabetes mellitus(T2DM) is a major disease with high penetrance in human around the world and still rising. This may cause other serious complications like kidney failure, heart failure, blindness etc. The early detection and diagnosis help to identify and may avoid these complications. Several classification algorithms exist but selecting the best classifier surely improves the accuracy of the predictions. The preprocessing methods selected in this study are Multiple Imputation, k-means for missing values treatment, Discretization to change in discrete values, Standard scaler, Min-Max scalar for feature scaling and, Random Forest (RF) for feature selection. For the classification Logistic Regression (LR), Artificial Neural Network (ANN), Support Vector Machine (SVM) and Random Forest(RF) are used. To evaluate the performance of model accuracy, sensitivity, specificity are used. This study compares the model performance with and without preprocessed data and has proved that the selected preprocessing methods significantly improves the model performance.Keywords
This publication has 17 references indexed in Scilit:
- A systematic map of medical data preprocessing in knowledge discoveryComputer Methods and Programs in Biomedicine, 2018
- Impact of selected pre-processing techniques on prediction of risk of early readmission for diabetic patients in IndiaInternational Journal of Diabetes in Developing Countries, 2016
- Tutorial on practical tips of the most influential data preprocessing algorithms in data miningKnowledge-Based Systems, 2016
- A neural network-based framework for the reconstruction of incomplete data setsNeurocomputing, 2010
- Data clustering: 50 years beyond K-meansPattern Recognition Letters, 2010
- Mining incomplete survey data through classificationKnowledge and Information Systems, 2009
- KEEL: a software tool to assess evolutionary algorithms for data mining problemsSoft Computing, 2008
- An analysis of four missing data treatment methods for supervised learningApplied Artificial Intelligence, 2003
- Discretization: An Enabling TechniqueData Mining and Knowledge Discovery, 2002
- An Introduction to Kernel and Nearest-Neighbor Nonparametric RegressionThe American Statistician, 1992