Overlap-Based Undersampling Method for Classification of Imbalanced Medical Datasets
- 29 May 2020
- book chapter
- Published by Springer Nature
- Vol. 584, 358-369
Abstract
Early diagnosis of some life-threatening diseases such as cancers and heart is crucial for effective treatments. Supervised machine learning has proved to be a very useful tool to serve this purpose. Historical data of patients including clinical and demographic information is used for training learning algorithms. This builds predictive models that provide initial diagnoses. However, in the medical domain, it is common to have the positive class under-represented in a dataset. In such a scenario, a typical learning algorithm tends to be biased towards the negative class, which is the majority class, and misclassify positive cases. This is known as the class imbalance problem. In this paper, a framework for predictive diagnostics of diseases with imbalanced records is presented. To reduce the classification bias, we propose the usage of an overlap-based undersampling method to improve the visibility of minority class samples in the region where the two classes overlap. This is achieved by detecting and removing negative class instances from the overlapping region. This will improve class separability in the data space. Experimental results show achievement of high accuracy in the positive class, which is highly preferable in the medical domain, while good trade-offs between sensitivity and specificity were obtained. Results also show that the method often outperformed other state-of-the-art and well-established techniques.Keywords
This publication has 22 references indexed in Scilit:
- A comprehensive data level analysis for cancer diagnosis on imbalanced dataJournal of Biomedical Informatics, 2019
- Distribution-Sensitive Unbalanced Data Oversampling Method for Medical DiagnosisJournal of Medical Systems, 2019
- A deep convolutional neural network model to classify heartbeatsComputers in Biology and Medicine, 2017
- Learning from class-imbalanced data: Review of methods and applicationsExpert Systems with Applications, 2017
- The study of under- and over-sampling methods’ utility in analysis of highly imbalanced data on osteoporosisInformation Sciences, 2017
- DBMUTE: density-based majority under-sampling techniqueKnowledge and Information Systems, 2016
- Polyp Detection via Imbalanced Learning and Discriminative Feature LearningIEEE Transactions on Medical Imaging, 2015
- DBSMOTE: Density-Based Synthetic Minority Over-sampling TEchniqueApplied Intelligence, 2011
- Borderline-SMOTE: A New Over-Sampling Method in Imbalanced Data Sets Learning2005
- SMOTE: Synthetic Minority Over-sampling TechniqueJournal of Artificial Intelligence Research, 2002