A generalized framework to predict continuous scores from medical ordinal labels
Authors:
- Katharina V. HoebelKHKatharina V. Hoebel
- Andreanne LemayALAndreanne Lemay
- John Peter CampbellJCJohn Peter Campbell
- Susan OstmoSOSusan Ostmo
- Michael F. ChiangMCMichael F. Chiang
- Christopher P. BridgeCBChristopher P. Bridge
- Matthew D. LiMLMatthew D. Li
- Praveer SinghPSPraveer Singh
- Aaron S. CoynerACAaron S. Coyner
- Jayashree Kalpathy-CramerJKJayashree Kalpathy-Cramer
Preprint
Abstract
Many variables of interest in clinical medicine, like disease severity, are recorded using discrete ordinal categories such as normal/mild/moderate/severe. These labels are used to train and evaluate disease severity prediction models. However, ordinal categories represent a simplification of an underlying continuous severity spectrum. Using continuous scores instead of ordinal categories is more sensitive to detecting small changes in disease severity over time. Here, we present a generalized framework that accurately predicts continuously valued variables using only discrete ordinal labels during model development. We found that for three clinical prediction tasks, models that take the ordinal relationship of the training labels into account outperformed conventional multi-class classification models. Particularly the continuous scores generated by ordinal classification and regression models showed a significantly higher correlation with expert rankings of disease severity and lower mean squared errors compared to the multi-class classification models. Furthermore, the use of MC dropout significantly improved the ability of all evaluated deep learning approaches to predict continuously valued scores that truthfully reflect the underlying continuous target variable. We showed that accurate continuously valued predictions can be generated even if the model development only involves discrete ordinal labels. The novel framework has been validated on three different clinical prediction tasks and has proven to bridge the gap between discrete ordinal labels and the underlying continuously valued variables.Keywords
All Related Versions
This publication has 27 references indexed in Scilit:
- Global, regional prevalence, incidence and risk factors of knee osteoarthritis in population-based studieseClinicalMedicine, 2020
- Monitoring Disease Progression With a Quantitative Severity Scale for Retinopathy of Prematurity Using Deep LearningJAMA Ophthalmology, 2019
- Performance of a Deep Learning Model vs Human Reviewers in Grading Endoscopic Disease Severity of Patients With Ulcerative ColitisJAMA Network Open, 2019
- Medical Image Analysis using Convolutional Neural Networks: A ReviewJournal of Medical Systems, 2018
- Automated Diagnosis of Plus Disease in Retinopathy of Prematurity Using Deep Convolutional Neural NetworksJAMA Ophthalmology, 2018
- Plus Disease in Retinopathy of Prematurity: Diagnostic Trends in 2016 Versus 2007American Journal of Ophthalmology, 2017
- Volumetric breast density affects performance of digital screening mammographyBreast Cancer Research and Treatment, 2016
- Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus PhotographsJAMA, 2016
- Robust Breast Composition Measurement - VolparaTM2010
- The International Classification of Retinopathy of Prematurity RevisitedArchives of Ophthalmology (1950), 2005