Practical considerations for specifying a super learner
Preprint
Abstract
Common tasks encountered in epidemiology, including disease incidence estimation and causal inference, rely on predictive modeling. Constructing a predictive model can be thought of as learning a prediction function, i.e., a function that takes as input covariate data and outputs a predicted value. Many strategies for learning these functions from data are available, from parametric regressions to machine learning algorithms. It can be challenging to choose an approach, as it is impossible to know in advance which one is the most suitable for a particular dataset and prediction task at hand. The super learner (SL) is an algorithm that alleviates concerns over selecting the one "right" strategy while providing the freedom to consider many of them, such as those recommended by collaborators, used in related research, or specified by subject-matter experts. It is an entirely pre-specified and data-adaptive strategy for predictive modeling. To ensure the SL is well-specified for learning the prediction function, the analyst does need to make a few important choices. In this Education Corner article, we provide step-by-step guidelines for making these choices, walking the reader through each of them and providing intuition along the way. In doing so, we aim to empower the analyst to tailor the SL specification to their prediction task, thereby ensuring their SL performs as well as possible. A flowchart provides a concise, easy-to-follow summary of key suggestions and heuristics, based on our accumulated experience, and guided by theory.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- Development and validation of an automated HIV prediction algorithm to identify candidates for pre-exposure prophylaxis: a modelling studyThe Lancet HIV, 2019
- Stacked generalization: an introduction to super learningEuropean Journal of Epidemiology, 2018
- Principled Machine Learning Using the Super Learner: An Application to Predicting Prison ViolenceSociological Methods & Research, 2018
- Constrained binary classification using ensemble learning: an application to cost‐efficient targeted PrEP strategiesStatistics in Medicine, 2017
- Improving Propensity Score Estimators' Robustness to Model Misspecification Using Super LearnerAmerican Journal of Epidemiology, 2014
- Super learning to hedge against incorrect inference from arbitrary parametric assumptions in marginal structural modelingJournal of Clinical Epidemiology, 2013
- Mortality Risk Score Prediction in an Elderly Population Using Machine LearningAmerican Journal of Epidemiology, 2013
- Asymptotics of cross-validated risk estimation in estimator selection and performance assessmentStatistical Methodology, 2005
- Regression Shrinkage and Selection Via the LassoJournal of the Royal Statistical Society Series B: Statistical Methodology, 1996
- Combining Possibly Related Estimation ProblemsJournal of the Royal Statistical Society Series B: Statistical Methodology, 1973