When the whole is greater than the sum of its parts: why machine learning and conventional statistics are complementary for predicting future health outcomes
Open Access
- 20 February 2025
- journal article
- review article
- Published by Oxford University Press (OUP) in Clinical Kidney Journal
- Vol. 18 (4) , sfaf059
Abstract
An artificial intelligence boom is currently ongoing, mainly due to large language models, leading to significant interest in artificial intelligence and subsequently also in machine learning (ML). One area where ML is often applied, prediction modelling, has also long been a focus of conventional statistics. As a result, multiple studies have aimed to prove superiority of one of the two scientific disciplines over the other. However, we argue that ML and conventional statistics should not be competing fields. Instead, both fields are intertwined and complementary to each other. To illustrate this, we discuss some essentials of prediction modelling, elaborate on prediction modelling using techniques from conventional statistics, and explain prediction modelling using common ML techniques such as support vector machines, random forests, and artificial neural networks. We then showcase that conventional statistics and ML are in fact similar in many aspects, including underlying statistical concepts and methods used in model development and validation. Finally, we argue that conventional statistics and ML can and should be seen as a single integrated field. This integration can further improve prediction modelling for both disciplines (e.g. regarding fairness and reporting standards) and will support the ultimate goal: developing the best performing prediction models for the patient and healthcare provider.Keywords
Funding Information
- Dutch Kidney Foundation (20OK016)
This publication has 52 references indexed in Scilit:
- Support vector methods for survival analysis: a comparison between ranking and regression approachesArtificial Intelligence in Medicine, 2011
- A Predictive Model for Progression of Chronic Kidney Disease to Kidney FailureJAMA, 2011
- A Comprehensive Risk Quantification Score for Deceased Donor Kidneys: The Kidney Donor Risk IndexTransplantation, 2009
- Prognostic Models With Competing RisksEpidemiology, 2009
- Prognosis and prognostic research: application and impact of prognostic models in clinical practiceBMJ, 2009
- Prognosis and prognostic research: validating a prognostic modelBMJ, 2009
- Prognosis and prognostic research: Developing a prognostic modelBMJ, 2009
- Dynamic Modeling and Statistical Analysis of Event TimesStatistical Science, 2006
- A new method of classifying prognostic comorbidity in longitudinal studies: Development and validationJournal of Chronic Diseases, 1987
- Evaluating the yield of medical testsJAMA, 1982