Rachael V. Phillips
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Rachael V. Phillips has a total of 69 co-authors. Here are the top co-authors by number of shared publications:
Mark J. van der Laan
Division of Biostatistics, School of Public Health, University of California, Berkeley, Berkeley, CA
7 shared publicationsAlan E. Hubbard
Division of Biostatistics University of California, Berkeley Berkeley California USA
5 shared publications- Mark J. van der Laan
University of California at Berkeley Berkeley California USA
5 shared publications - Alan E. Hubbard
School of Public Health, University of California, Berkeley, Berkeley, CA USA
4 shared publications - Jeremy Coyle
Division of Epidemiology & Biostatistics, University of California, Berkeley, Berkeley, CA, USA
4 shared publications - Nima S. Hejazi
Department of Biostatistics, Harvard University, Cambridge, Massachusetts, USA
4 shared publications
Latest Publications
- Open Access
- 22 October 2026
- Published by Wiley in Cancer Medicine
- Vol. 15 (10) , e72348
Core Self‐Reported Variables for Next‐Generation Colorectal Cancer Risk Prediction: An International Expert Consensus Study
Background: Colorectal cancer remains a major global health burden, and although population‐based screening is beneficial, current programs have notable limitations. Risk‐stratified screening has been proposed, yet existing prediction models vary in performance and in the inclusion of predictors. Data‐adaptive modeling (machine learning) may strengthen risk prediction but depends on high‐quality input data. Expert consensus methods offer a structured way to identify clinically meaningful variables for future tools. Objective: To establish expert consensus on the selection of relevant variables and refinement of self‐reported questionnaire items for next‐generation colorectal cancer risk‐prediction tools. Methods: A modified Delphi study was conducted with seventeen internationally recognized experts. The process consisted of anonymized, email‐based rounds aimed at refining a questionnaire for colorectal cancer risk prediction. Across three iterative rounds, experts rated item relevance, proposed revisions, and identified missing domains while reviewing group‐level summaries of ratings and comments. Items not meeting predefined thresholds were removed, and validated lifestyle‐related items were incorporated. Consensus was defined as 15 of 17 experts rating an item within a similar three‐point range. Results: All seventeen experts completed all rounds. The final I‐CRC‐RISK Questionnaire comprises 23 items across seven categories: Demography , Health Status, Family Health History, Dietary Patterns, Physical Activity, Smoking , and Alcohol Consumption . Expert consensus was reached for 19 items. Conclusion: The I ‐ CRC ‐ RISK Questionnaire provides an expert‐informed set of self‐reported variables to support future risk prediction efforts. It offers a structured basis for further validation and psychometric testing before prospective data collection may be undertaken. This foundational work could enhance the predictive performance of both traditional and machine‐learning approaches by improving the consistency and relevance of input data.
- 22 October 2026
- Published by Wiley in Cancer Medicine
- Vol. 15 (10) , e72348
Open AccessCore Self‐Reported Variables for Next‐Generation Colorectal Cancer Risk Prediction: An International Expert Consensus Study
Background: Colorectal cancer remains a major global health burden, and although population‐based screening is beneficial, current programs have notable limitations. Risk‐stratified screening has been proposed, yet existing prediction models vary in performance and in the inclusion of predictors. Data‐adaptive modeling (machine learning) may strengthen risk prediction but depends on high‐quality input data. Expert consensus methods offer a structured way to identify clinically meaningful variables for future tools. Objective: To establish expert consensus on the selection of relevant variables and refinement of self‐reported questionnaire items for next‐generation colorectal cancer risk‐prediction tools. Methods: A modified Delphi study was conducted with seventeen internationally recognized experts. The process consisted of anonymized, email‐based rounds aimed at refining a questionnaire for colorectal cancer risk prediction. Across three iterative rounds, experts rated item relevance, proposed revisions, and identified missing domains while reviewing group‐level summaries of ratings and comments. Items not meeting predefined thresholds were removed, and validated lifestyle‐related items were incorporated. Consensus was defined as 15 of 17 experts rating an item within a similar three‐point range. Results: All seventeen experts completed all rounds. The final I‐CRC‐RISK Questionnaire comprises 23 items across seven categories: Demography , Health Status, Family Health History, Dietary Patterns, Physical Activity, Smoking , and Alcohol Consumption . Expert consensus was reached for 19 items. Conclusion: The I ‐ CRC ‐ RISK Questionnaire provides an expert‐informed set of self‐reported variables to support future risk prediction efforts. It offers a structured basis for further validation and psychometric testing before prospective data collection may be undertaken. This foundational work could enhance the predictive performance of both traditional and machine‐learning approaches by improving the consistency and relevance of input data.
- Open Access
- 1 March 2025
- Published by Project MUSE in Observational Studies
- Vol. 11 (1) , 61-75
Commentary on “Nonparametric identification is not enough, but randomized controlled trials are”: Statistical considerations for generating reliable evidence across a spectrum of studies that increasingly involve real-world elements
Judea Pearl, quoted in Pearl and Mackenzie (2008), stated that “once we have understood why [randomized controlled trials] RCTs work, there is no need to put them on a pedestal and treat them as the gold standard of causal analysis, which all other methods should emulate.” In Aronow et al. (2024), this claim is refuted, drawing on results of Robins and Ritov (1997). The argument is made that statistical estimation and inference tend to be fundamentally more difficult in observational studies than in randomized controlled trials, even when all confounders are observed and measured without error. We congratulate the authors for raising this highly timely, interesting discussion and welcome this opportunity to join this important debate. In this commentary, we focus on what it takes to generate reliable evidence across a spectrum of studies that increasingly involve real-world elements and less control over design. A related question is whether, along this spectrum of studies, the reliability of evidence generated by a statistical analysis decreases. We claim that this is not the case, but that the challenge for the appropriate statistical method increases, requiring sophisticated and careful execution.
- 1 March 2025
- Published by Project MUSE in Observational Studies
- Vol. 11 (1) , 61-75
Open AccessCommentary on “Nonparametric identification is not enough, but randomized controlled trials are”: Statistical considerations for generating reliable evidence across a spectrum of studies that increasingly involve real-world elements
Judea Pearl, quoted in Pearl and Mackenzie (2008), stated that “once we have understood why [randomized controlled trials] RCTs work, there is no need to put them on a pedestal and treat them as the gold standard of causal analysis, which all other methods should emulate.” In Aronow et al. (2024), this claim is refuted, drawing on results of Robins and Ritov (1997). The argument is made that statistical estimation and inference tend to be fundamentally more difficult in observational studies than in randomized controlled trials, even when all confounders are observed and measured without error. We congratulate the authors for raising this highly timely, interesting discussion and welcome this opportunity to join this important debate. In this commentary, we focus on what it takes to generate reliable evidence across a spectrum of studies that increasingly involve real-world elements and less control over design. A related question is whether, along this spectrum of studies, the reliability of evidence generated by a statistical analysis decreases. We claim that this is not the case, but that the challenge for the appropriate statistical method increases, requiring sophisticated and careful execution.
- Open Access
[Invited Discussion] Randomization Tests to Address Disruptions in Clinical Trials: A Report from the NISS Ingram Olkin Forum Series on Unplanned Clinical Trial Disruptions
Disruptions in clinical trials may be due to external events like pandemics, warfare, and natural disasters. Resulting complications may lead to unforeseen intercurrent events (events that occur after treatment initiation and affect the interpretation of the clinical question of interest or the existence of the measurements associated with it). In Uschner et al. (2023), several example clinical trial disruptions are described: treatment effect drift, population shift, change of care, change of data collection, and change of availability of study medication. A complex randomized controlled trial (RCT) setting with (planned or unplanned) intercurrent events is then described, and randomization tests are presented as a means for non-parametric inference that is robust to violations of assumption typically made in clinical trials. While estimation methods like Targeted Learning (TL) are valid in such settings, we do not see where the authors make the case that one should be going for a randomization test in such disrupted RCTs. In this discussion, we comment on the appropriateness of TL and the accompanying TL Roadmap in the context of disrupted clinical trials. We highlight a few key articles related to the broad applicability of TL for RCTs and real-world data (RWD) analyses with intercurrent events. We begin by introducing TL and motivating its utility in Section 2, and then in Section 3 we provide a brief overview of the TL Roadmap. In Section 4 we recite the example clinical trial disruptions presented in Uschner et al. (2023), discussing considerations and solutions based on the principles of TL. We request in an authors' rejoinder a clear theoretical demonstration with specific examples in this setting that a randomization test is the only valid inferential method relative to one based on following the TL Roadmap.
Open Access[Invited Discussion] Randomization Tests to Address Disruptions in Clinical Trials: A Report from the NISS Ingram Olkin Forum Series on Unplanned Clinical Trial Disruptions
Disruptions in clinical trials may be due to external events like pandemics, warfare, and natural disasters. Resulting complications may lead to unforeseen intercurrent events (events that occur after treatment initiation and affect the interpretation of the clinical question of interest or the existence of the measurements associated with it). In Uschner et al. (2023), several example clinical trial disruptions are described: treatment effect drift, population shift, change of care, change of data collection, and change of availability of study medication. A complex randomized controlled trial (RCT) setting with (planned or unplanned) intercurrent events is then described, and randomization tests are presented as a means for non-parametric inference that is robust to violations of assumption typically made in clinical trials. While estimation methods like Targeted Learning (TL) are valid in such settings, we do not see where the authors make the case that one should be going for a randomization test in such disrupted RCTs. In this discussion, we comment on the appropriateness of TL and the accompanying TL Roadmap in the context of disrupted clinical trials. We highlight a few key articles related to the broad applicability of TL for RCTs and real-world data (RWD) analyses with intercurrent events. We begin by introducing TL and motivating its utility in Section 2, and then in Section 3 we provide a brief overview of the TL Roadmap. In Section 4 we recite the example clinical trial disruptions presented in Uschner et al. (2023), discussing considerations and solutions based on the principles of TL. We request in an authors' rejoinder a clear theoretical demonstration with specific examples in this setting that a randomization test is the only valid inferential method relative to one based on following the TL Roadmap.
- Open Access
- 15 August 2024
- Published by JMIR Publications Inc. in JMIR Public Health and Surveillance
- Vol. 10, e53322
Predicting Long COVID in the National COVID Cohort Collaborative Using Super Learner: Cohort Study
Background: Postacute sequelae of COVID-19 (PASC), also known as long COVID, is a broad grouping of a range of long-term symptoms following acute COVID-19. These symptoms can occur across a range of biological systems, leading to challenges in determining risk factors for PASC and the causal etiology of this disorder. An understanding of characteristics that are predictive of future PASC is valuable, as this can inform the identification of high-risk individuals and future preventative efforts. However, current knowledge regarding PASC risk factors is limited. Objective: Using a sample of 55,257 patients (at a ratio of 1 patient with PASC to 4 matched controls) from the National COVID Cohort Collaborative, as part of the National Institutes of Health Long COVID Computational Challenge, we sought to predict individual risk of PASC diagnosis from a curated set of clinically informed covariates. The National COVID Cohort Collaborative includes electronic health records for more than 22 million patients from 84 sites across the United States. Methods: We predicted individual PASC status, given covariate information, using Super Learner (an ensemble machine learning algorithm also known as stacking) to learn the optimal combination of gradient boosting and random forest algorithms to maximize the area under the receiver operator curve. We evaluated variable importance (Shapley values) based on 3 levels: individual features, temporal windows, and clinical domains. We externally validated these findings using a holdout set of randomly selected study sites. Results: We were able to predict individual PASC diagnoses accurately (area under the curve 0.874). The individual features of the length of observation period, number of health care interactions during acute COVID-19, and viral lower respiratory infection were the most predictive of subsequent PASC diagnosis. Temporally, we found that baseline characteristics were the most predictive of future PASC diagnosis, compared with characteristics immediately before, during, or after acute COVID-19. We found that the clinical domains of health care use, demographics or anthropometry, and respiratory factors were the most predictive of PASC diagnosis. Conclusions: The methods outlined here provide an open-source, applied example of using Super Learner to predict PASC status using electronic health record data, which can be replicated across a variety of settings. Across individual predictors and clinical domains, we consistently found that factors related to health care use were the strongest predictors of PASC diagnosis. This indicates that any observational studies using PASC diagnosis as a primary outcome must rigorously account for heterogeneous health care use. Our temporal findings support the hypothesis that clinicians may be able to accurately assess the risk of PASC in patients before acute COVID-19 diagnosis, which could improve early interventions and preventive care. Our findings also highlight the importance of respiratory characteristics in PASC risk assessment. International Registered Report Identifier (IRRID): RR2-10.1101/2023.07.27.23293272
- 15 August 2024
- Published by JMIR Publications Inc. in JMIR Public Health and Surveillance
- Vol. 10, e53322
Open AccessPredicting Long COVID in the National COVID Cohort Collaborative Using Super Learner: Cohort Study
Background: Postacute sequelae of COVID-19 (PASC), also known as long COVID, is a broad grouping of a range of long-term symptoms following acute COVID-19. These symptoms can occur across a range of biological systems, leading to challenges in determining risk factors for PASC and the causal etiology of this disorder. An understanding of characteristics that are predictive of future PASC is valuable, as this can inform the identification of high-risk individuals and future preventative efforts. However, current knowledge regarding PASC risk factors is limited. Objective: Using a sample of 55,257 patients (at a ratio of 1 patient with PASC to 4 matched controls) from the National COVID Cohort Collaborative, as part of the National Institutes of Health Long COVID Computational Challenge, we sought to predict individual risk of PASC diagnosis from a curated set of clinically informed covariates. The National COVID Cohort Collaborative includes electronic health records for more than 22 million patients from 84 sites across the United States. Methods: We predicted individual PASC status, given covariate information, using Super Learner (an ensemble machine learning algorithm also known as stacking) to learn the optimal combination of gradient boosting and random forest algorithms to maximize the area under the receiver operator curve. We evaluated variable importance (Shapley values) based on 3 levels: individual features, temporal windows, and clinical domains. We externally validated these findings using a holdout set of randomly selected study sites. Results: We were able to predict individual PASC diagnoses accurately (area under the curve 0.874). The individual features of the length of observation period, number of health care interactions during acute COVID-19, and viral lower respiratory infection were the most predictive of subsequent PASC diagnosis. Temporally, we found that baseline characteristics were the most predictive of future PASC diagnosis, compared with characteristics immediately before, during, or after acute COVID-19. We found that the clinical domains of health care use, demographics or anthropometry, and respiratory factors were the most predictive of PASC diagnosis. Conclusions: The methods outlined here provide an open-source, applied example of using Super Learner to predict PASC status using electronic health record data, which can be replicated across a variety of settings. Across individual predictors and clinical domains, we consistently found that factors related to health care use were the strongest predictors of PASC diagnosis. This indicates that any observational studies using PASC diagnosis as a primary outcome must rigorously account for heterogeneous health care use. Our temporal findings support the hypothesis that clinicians may be able to accurately assess the risk of PASC in patients before acute COVID-19 diagnosis, which could improve early interventions and preventive care. Our findings also highlight the importance of respiratory characteristics in PASC risk assessment. International Registered Report Identifier (IRRID): RR2-10.1101/2023.07.27.23293272
- Open Access
- 3 October 2023
- Published by JMIR Publications Inc. in JMIR Preprints
Predicting Long COVID in the National COVID Cohort Collaborative Using Super Learner: Cohort Study (Preprint)
BACKGROUND Postacute sequelae of COVID-19 (PASC), also known as long COVID, is a broad grouping of a range of long-term symptoms following acute COVID-19. These symptoms can occur across a range of biological systems, leading to challenges in determining risk factors for PASC and the causal etiology of this disorder. An understanding of characteristics that are predictive of future PASC is valuable, as this can inform the identification of high-risk individuals and future preventative efforts. However, current knowledge regarding PASC risk factors is limited. OBJECTIVE Using a sample of 55,257 patients (at a ratio of 1 patient with PASC to 4 matched controls) from the National COVID Cohort Collaborative, as part of the National Institutes of Health Long COVID Computational Challenge, we sought to predict individual risk of PASC diagnosis from a curated set of clinically informed covariates. The National COVID Cohort Collaborative includes electronic health records for more than 22 million patients from 84 sites across the United States. METHODS We predicted individual PASC status, given covariate information, using Super Learner (an ensemble machine learning algorithm also known as stacking) to learn the optimal combination of gradient boosting and random forest algorithms to maximize the area under the receiver operator curve. We evaluated variable importance (Shapley values) based on 3 levels: individual features, temporal windows, and clinical domains. We externally validated these findings using a holdout set of randomly selected study sites. RESULTS We were able to predict individual PASC diagnoses accurately (area under the curve 0.874). The individual features of the length of observation period, number of health care interactions during acute COVID-19, and viral lower respiratory infection were the most predictive of subsequent PASC diagnosis. Temporally, we found that baseline characteristics were the most predictive of future PASC diagnosis, compared with characteristics immediately before, during, or after acute COVID-19. We found that the clinical domains of health care use, demographics or anthropometry, and respiratory factors were the most predictive of PASC diagnosis. CONCLUSIONS The methods outlined here provide an open-source, applied example of using Super Learner to predict PASC status using electronic health record data, which can be replicated across a variety of settings. Across individual predictors and clinical domains, we consistently found that factors related to health care use were the strongest predictors of PASC diagnosis. This indicates that any observational studies using PASC diagnosis as a primary outcome must rigorously account for heterogeneous health care use. Our temporal findings support the hypothesis that clinicians may be able to accurately assess the risk of PASC in patients before acute COVID-19 diagnosis, which could improve early interventions and preventive care. Our findings also highlight the importance of respiratory characteristics in PASC risk assessment. INTERNATIONAL REGISTERED REPORT RR2-10.1101/2023.07.27.23293272
- 3 October 2023
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessPredicting Long COVID in the National COVID Cohort Collaborative Using Super Learner: Cohort Study (Preprint)
BACKGROUND Postacute sequelae of COVID-19 (PASC), also known as long COVID, is a broad grouping of a range of long-term symptoms following acute COVID-19. These symptoms can occur across a range of biological systems, leading to challenges in determining risk factors for PASC and the causal etiology of this disorder. An understanding of characteristics that are predictive of future PASC is valuable, as this can inform the identification of high-risk individuals and future preventative efforts. However, current knowledge regarding PASC risk factors is limited. OBJECTIVE Using a sample of 55,257 patients (at a ratio of 1 patient with PASC to 4 matched controls) from the National COVID Cohort Collaborative, as part of the National Institutes of Health Long COVID Computational Challenge, we sought to predict individual risk of PASC diagnosis from a curated set of clinically informed covariates. The National COVID Cohort Collaborative includes electronic health records for more than 22 million patients from 84 sites across the United States. METHODS We predicted individual PASC status, given covariate information, using Super Learner (an ensemble machine learning algorithm also known as stacking) to learn the optimal combination of gradient boosting and random forest algorithms to maximize the area under the receiver operator curve. We evaluated variable importance (Shapley values) based on 3 levels: individual features, temporal windows, and clinical domains. We externally validated these findings using a holdout set of randomly selected study sites. RESULTS We were able to predict individual PASC diagnoses accurately (area under the curve 0.874). The individual features of the length of observation period, number of health care interactions during acute COVID-19, and viral lower respiratory infection were the most predictive of subsequent PASC diagnosis. Temporally, we found that baseline characteristics were the most predictive of future PASC diagnosis, compared with characteristics immediately before, during, or after acute COVID-19. We found that the clinical domains of health care use, demographics or anthropometry, and respiratory factors were the most predictive of PASC diagnosis. CONCLUSIONS The methods outlined here provide an open-source, applied example of using Super Learner to predict PASC status using electronic health record data, which can be replicated across a variety of settings. Across individual predictors and clinical domains, we consistently found that factors related to health care use were the strongest predictors of PASC diagnosis. This indicates that any observational studies using PASC diagnosis as a primary outcome must rigorously account for heterogeneous health care use. Our temporal findings support the hypothesis that clinicians may be able to accurately assess the risk of PASC in patients before acute COVID-19 diagnosis, which could improve early interventions and preventive care. Our findings also highlight the importance of respiratory characteristics in PASC risk assessment. INTERNATIONAL REGISTERED REPORT RR2-10.1101/2023.07.27.23293272
- Open Access
Practical considerations for specifying a super learner
Common tasks encountered in epidemiology, including disease incidence estimation and causal inference, rely on predictive modeling. Constructing a predictive model can be thought of as learning a prediction function, i.e., a function that takes as input covariate data and outputs a predicted value. Many strategies for learning these functions from data are available, from parametric regressions to machine learning algorithms. It can be challenging to choose an approach, as it is impossible to know in advance which one is the most suitable for a particular dataset and prediction task at hand. The super learner (SL) is an algorithm that alleviates concerns over selecting the one "right" strategy while providing the freedom to consider many of them, such as those recommended by collaborators, used in related research, or specified by subject-matter experts. It is an entirely pre-specified and data-adaptive strategy for predictive modeling. To ensure the SL is well-specified for learning the prediction function, the analyst does need to make a few important choices. In this Education Corner article, we provide step-by-step guidelines for making these choices, walking the reader through each of them and providing intuition along the way. In doing so, we aim to empower the analyst to tailor the SL specification to their prediction task, thereby ensuring their SL performs as well as possible. A flowchart provides a concise, easy-to-follow summary of key suggestions and heuristics, based on our accumulated experience, and guided by theory.
Open AccessPractical considerations for specifying a super learner
Common tasks encountered in epidemiology, including disease incidence estimation and causal inference, rely on predictive modeling. Constructing a predictive model can be thought of as learning a prediction function, i.e., a function that takes as input covariate data and outputs a predicted value. Many strategies for learning these functions from data are available, from parametric regressions to machine learning algorithms. It can be challenging to choose an approach, as it is impossible to know in advance which one is the most suitable for a particular dataset and prediction task at hand. The super learner (SL) is an algorithm that alleviates concerns over selecting the one "right" strategy while providing the freedom to consider many of them, such as those recommended by collaborators, used in related research, or specified by subject-matter experts. It is an entirely pre-specified and data-adaptive strategy for predictive modeling. To ensure the SL is well-specified for learning the prediction function, the analyst does need to make a few important choices. In this Education Corner article, we provide step-by-step guidelines for making these choices, walking the reader through each of them and providing intuition along the way. In doing so, we aim to empower the analyst to tailor the SL specification to their prediction task, thereby ensuring their SL performs as well as possible. A flowchart provides a concise, easy-to-follow summary of key suggestions and heuristics, based on our accumulated experience, and guided by theory.
- Top Cited PapersOpen Access
- 11 March 2023
- Published by Oxford University Press (OUP) in International Journal of Epidemiology
- Vol. 52 (4) , 1276-1285
Practical considerations for specifying a super learner
Common tasks encountered in epidemiology, including disease incidence estimation and causal inference, rely on predictive modelling. Constructing a predictive model can be thought of as learning a prediction function (a function that takes as input covariate data and outputs a predicted value). Many strategies for learning prediction functions from data (learners) are available, from parametric regressions to machine learning algorithms. It can be challenging to choose a learner, as it is impossible to know in advance which one is the most suitable for a particular dataset and prediction task. The super learner (SL) is an algorithm that alleviates concerns over selecting the one ‘right’ learner by providing the freedom to consider many, such as those recommended by collaborators, used in related research or specified by subject-matter experts. Also known as stacking, SL is an entirely prespecified and flexible approach for predictive modelling. To ensure the SL is well specified for learning the desired prediction function, the analyst does need to make a few important choices. In this educational article, we provide step-by-step guidelines for making these decisions, walking the reader through each of them and providing intuition along the way. In doing so, we aim to empower the analyst to tailor the SL specification to their prediction task, thereby ensuring their SL performs as well as possible. A flowchart provides a concise, easy-to-follow summary of key suggestions and heuristics, based on our accumulated experience and guided by SL optimality theory.
- 11 March 2023
- Published by Oxford University Press (OUP) in International Journal of Epidemiology
- Vol. 52 (4) , 1276-1285
Top Cited PapersOpen AccessPractical considerations for specifying a super learner
Common tasks encountered in epidemiology, including disease incidence estimation and causal inference, rely on predictive modelling. Constructing a predictive model can be thought of as learning a prediction function (a function that takes as input covariate data and outputs a predicted value). Many strategies for learning prediction functions from data (learners) are available, from parametric regressions to machine learning algorithms. It can be challenging to choose a learner, as it is impossible to know in advance which one is the most suitable for a particular dataset and prediction task. The super learner (SL) is an algorithm that alleviates concerns over selecting the one ‘right’ learner by providing the freedom to consider many, such as those recommended by collaborators, used in related research or specified by subject-matter experts. Also known as stacking, SL is an entirely prespecified and flexible approach for predictive modelling. To ensure the SL is well specified for learning the desired prediction function, the analyst does need to make a few important choices. In this educational article, we provide step-by-step guidelines for making these decisions, walking the reader through each of them and providing intuition along the way. In doing so, we aim to empower the analyst to tailor the SL specification to their prediction task, thereby ensuring their SL performs as well as possible. A flowchart provides a concise, easy-to-follow summary of key suggestions and heuristics, based on our accumulated experience and guided by SL optimality theory.
- 16 January 2023
- Published by Wiley in Statistics in Medicine
- Vol. 42 (7) , 1013-1044
Personalized online ensemble machine learning with applications for dynamic data streams
In this work we introduce the personalized online super learner (POSL), an online personalizable ensemble machine learning algorithm for streaming data. POSL optimizes predictions with respect to baseline covariates, so personalization can vary from completely individualized, that is, optimization with respect to subject ID, to many individuals, that is, optimization with respect to common baseline covariates. As an online algorithm, POSL learns in real time. As a super learner, POSL is grounded in statistical optimality theory and can leverage a diversity of candidate algorithms, including online algorithms with different training and update times, fixed/offline algorithms that are not updated during POSL's fitting procedure, pooled algorithms that learn from many individuals' time series, and individualized algorithms that learn from within a single time series. POSL's ensembling of the candidates can depend on the amount of data collected, the stationarity of the time series, and the mutual characteristics of a group of time series. Depending on the underlying data‐generating process and the information available in the data, POSL is able to adapt to learning across samples, through time, or both. For a range of simulations that reflect realistic forecasting scenarios and in a medical application, we examine the performance of POSL relative to other current ensembling and online learning methods. We show that POSL is able to provide reliable predictions for both short and long time series, and it's able to adjust to changing data‐generating environments. We further cultivate POSL's practicality by extending it to settings where time series dynamically enter and exit.
- 16 January 2023
- Published by Wiley in Statistics in Medicine
- Vol. 42 (7) , 1013-1044
Personalized online ensemble machine learning with applications for dynamic data streams
In this work we introduce the personalized online super learner (POSL), an online personalizable ensemble machine learning algorithm for streaming data. POSL optimizes predictions with respect to baseline covariates, so personalization can vary from completely individualized, that is, optimization with respect to subject ID, to many individuals, that is, optimization with respect to common baseline covariates. As an online algorithm, POSL learns in real time. As a super learner, POSL is grounded in statistical optimality theory and can leverage a diversity of candidate algorithms, including online algorithms with different training and update times, fixed/offline algorithms that are not updated during POSL's fitting procedure, pooled algorithms that learn from many individuals' time series, and individualized algorithms that learn from within a single time series. POSL's ensembling of the candidates can depend on the amount of data collected, the stationarity of the time series, and the mutual characteristics of a group of time series. Depending on the underlying data‐generating process and the information available in the data, POSL is able to adapt to learning across samples, through time, or both. For a range of simulations that reflect realistic forecasting scenarios and in a medical application, we examine the performance of POSL relative to other current ensembling and online learning methods. We show that POSL is able to provide reliable predictions for both short and long time series, and it's able to adjust to changing data‐generating environments. We further cultivate POSL's practicality by extending it to settings where time series dynamically enter and exit.
- Open Access
- 1 July 2022
- Published by Oxford University Press (OUP) in Journal of the Royal Statistical Society Series B: Statistical Methodology
- Vol. 84 (3) , 717-718
Rachael V. Phillips and Mark J. van der Laan’s Contribution to the Discussion of ‘Assumption-Lean Inference for Generalised Linear Model Parameters’ by Vansteelandt and Dukes
- 1 July 2022
- Published by Oxford University Press (OUP) in Journal of the Royal Statistical Society Series B: Statistical Methodology
- Vol. 84 (3) , 717-718
Open AccessRachael V. Phillips and Mark J. van der Laan’s Contribution to the Discussion of ‘Assumption-Lean Inference for Generalised Linear Model Parameters’ by Vansteelandt and Dukes