RPRachael V. Phillips

Rachael V. Phillips

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Rachael V. Phillips has a total of 69 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Core Self‐Reported Variables for Next‐Generation Colorectal Cancer Risk Prediction: An International Expert Consensus Study

    by Hanna Dubois, Eva Torbjörnsson, Jessica Norrbom, Alan Hubbard, Mirna Enskär, Rachael V. Phillips, Mark van der Laan, Johannes Blom

    Background: Colorectal cancer remains a major global health burden, and although population‐based screening is beneficial, current programs have notable limitations. Risk‐stratified screening has been proposed, yet existing prediction models vary in performance and in the inclusion of predictors. Data‐adaptive modeling (machine learning) may strengthen risk prediction but depends on high‐quality input data. Expert consensus methods offer a structured way to identify clinically meaningful variables for future tools. Objective: To establish expert consensus on the selection of relevant variables and refinement of self‐reported questionnaire items for next‐generation colorectal cancer risk‐prediction tools. Methods: A modified Delphi study was conducted with seventeen internationally recognized experts. The process consisted of anonymized, email‐based rounds aimed at refining a questionnaire for colorectal cancer risk prediction. Across three iterative rounds, experts rated item relevance, proposed revisions, and identified missing domains while reviewing group‐level summaries of ratings and comments. Items not meeting predefined thresholds were removed, and validated lifestyle‐related items were incorporated. Consensus was defined as 15 of 17 experts rating an item within a similar three‐point range. Results: All seventeen experts completed all rounds. The final I‐CRC‐RISK Questionnaire comprises 23 items across seven categories: Demography , Health Status, Family Health History, Dietary Patterns, Physical Activity, Smoking , and Alcohol Consumption . Expert consensus was reached for 19 items. Conclusion: The I ‐ CRC ‐ RISK Questionnaire provides an expert‐informed set of self‐reported variables to support future risk prediction efforts. It offers a structured basis for further validation and psychometric testing before prospective data collection may be undertaken. This foundational work could enhance the predictive performance of both traditional and machine‐learning approaches by improving the consistency and relevance of input data.

  • Open Access

    Commentary on “Nonparametric identification is not enough, but randomized controlled trials are”: Statistical considerations for generating reliable evidence across a spectrum of studies that increasingly involve real-world elements

    by Rachael V. Phillips, Mark J. van der Laan

    Judea Pearl, quoted in Pearl and Mackenzie (2008), stated that “once we have understood why [randomized controlled trials] RCTs work, there is no need to put them on a pedestal and treat them as the gold standard of causal analysis, which all other methods should emulate.” In Aronow et al. (2024), this claim is refuted, drawing on results of Robins and Ritov (1997). The argument is made that statistical estimation and inference tend to be fundamentally more difficult in observational studies than in randomized controlled trials, even when all confounders are observed and measured without error. We congratulate the authors for raising this highly timely, interesting discussion and welcome this opportunity to join this important debate. In this commentary, we focus on what it takes to generate reliable evidence across a spectrum of studies that increasingly involve real-world elements and less control over design. A related question is whether, along this spectrum of studies, the reliability of evidence generated by a statistical analysis decreases. We claim that this is not the case, but that the challenge for the appropriate statistical method increases, requiring sophisticated and careful execution.


  • Open Access

    [Invited Discussion] Randomization Tests to Address Disruptions in Clinical Trials: A Report from the NISS Ingram Olkin Forum Series on Unplanned Clinical Trial Disruptions

    by Rachael V. Phillips, Mark J. van der Laan

    Disruptions in clinical trials may be due to external events like pandemics, warfare, and natural disasters. Resulting complications may lead to unforeseen intercurrent events (events that occur after treatment initiation and affect the interpretation of the clinical question of interest or the existence of the measurements associated with it). In Uschner et al. (2023), several example clinical trial disruptions are described: treatment effect drift, population shift, change of care, change of data collection, and change of availability of study medication. A complex randomized controlled trial (RCT) setting with (planned or unplanned) intercurrent events is then described, and randomization tests are presented as a means for non-parametric inference that is robust to violations of assumption typically made in clinical trials. While estimation methods like Targeted Learning (TL) are valid in such settings, we do not see where the authors make the case that one should be going for a randomization test in such disrupted RCTs. In this discussion, we comment on the appropriateness of TL and the accompanying TL Roadmap in the context of disrupted clinical trials. We highlight a few key articles related to the broad applicability of TL for RCTs and real-world data (RWD) analyses with intercurrent events. We begin by introducing TL and motivating its utility in Section 2, and then in Section 3 we provide a brief overview of the TL Roadmap. In Section 4 we recite the example clinical trial disruptions presented in Uschner et al. (2023), discussing considerations and solutions based on the principles of TL. We request in an authors' rejoinder a clear theoretical demonstration with specific examples in this setting that a randomization test is the only valid inferential method relative to one based on following the TL Roadmap.

  • Open Access

    Predicting Long COVID in the National COVID Cohort Collaborative Using Super Learner: Cohort Study

    by Zachary Butzin-Dozier, Yunwen Ji, Haodong Li, Jeremy Coyle, Junming Shi, Rachael V Phillips, Andrew N Mertens, Romain Pirracchio, Mark J van der Laan, Rena C Patel, et al.

    Background: Postacute sequelae of COVID-19 (PASC), also known as long COVID, is a broad grouping of a range of long-term symptoms following acute COVID-19. These symptoms can occur across a range of biological systems, leading to challenges in determining risk factors for PASC and the causal etiology of this disorder. An understanding of characteristics that are predictive of future PASC is valuable, as this can inform the identification of high-risk individuals and future preventative efforts. However, current knowledge regarding PASC risk factors is limited. Objective: Using a sample of 55,257 patients (at a ratio of 1 patient with PASC to 4 matched controls) from the National COVID Cohort Collaborative, as part of the National Institutes of Health Long COVID Computational Challenge, we sought to predict individual risk of PASC diagnosis from a curated set of clinically informed covariates. The National COVID Cohort Collaborative includes electronic health records for more than 22 million patients from 84 sites across the United States. Methods: We predicted individual PASC status, given covariate information, using Super Learner (an ensemble machine learning algorithm also known as stacking) to learn the optimal combination of gradient boosting and random forest algorithms to maximize the area under the receiver operator curve. We evaluated variable importance (Shapley values) based on 3 levels: individual features, temporal windows, and clinical domains. We externally validated these findings using a holdout set of randomly selected study sites. Results: We were able to predict individual PASC diagnoses accurately (area under the curve 0.874). The individual features of the length of observation period, number of health care interactions during acute COVID-19, and viral lower respiratory infection were the most predictive of subsequent PASC diagnosis. Temporally, we found that baseline characteristics were the most predictive of future PASC diagnosis, compared with characteristics immediately before, during, or after acute COVID-19. We found that the clinical domains of health care use, demographics or anthropometry, and respiratory factors were the most predictive of PASC diagnosis. Conclusions: The methods outlined here provide an open-source, applied example of using Super Learner to predict PASC status using electronic health record data, which can be replicated across a variety of settings. Across individual predictors and clinical domains, we consistently found that factors related to health care use were the strongest predictors of PASC diagnosis. This indicates that any observational studies using PASC diagnosis as a primary outcome must rigorously account for heterogeneous health care use. Our temporal findings support the hypothesis that clinicians may be able to accurately assess the risk of PASC in patients before acute COVID-19 diagnosis, which could improve early interventions and preventive care. Our findings also highlight the importance of respiratory characteristics in PASC risk assessment. International Registered Report Identifier (IRRID): RR2-10.1101/2023.07.27.23293272


  • Open Access

    Predicting Long COVID in the National COVID Cohort Collaborative Using Super Learner: Cohort Study (Preprint)

    by Zachary Butzin-Dozier, Yunwen Ji, Haodong Li, Jeremy Coyle, Junming Shi, Rachael V Phillips, Andrew N Mertens, Romain Pirracchio, Mark J van der Laan, Rena C Patel, et al.

    BACKGROUND Postacute sequelae of COVID-19 (PASC), also known as long COVID, is a broad grouping of a range of long-term symptoms following acute COVID-19. These symptoms can occur across a range of biological systems, leading to challenges in determining risk factors for PASC and the causal etiology of this disorder. An understanding of characteristics that are predictive of future PASC is valuable, as this can inform the identification of high-risk individuals and future preventative efforts. However, current knowledge regarding PASC risk factors is limited. OBJECTIVE Using a sample of 55,257 patients (at a ratio of 1 patient with PASC to 4 matched controls) from the National COVID Cohort Collaborative, as part of the National Institutes of Health Long COVID Computational Challenge, we sought to predict individual risk of PASC diagnosis from a curated set of clinically informed covariates. The National COVID Cohort Collaborative includes electronic health records for more than 22 million patients from 84 sites across the United States. METHODS We predicted individual PASC status, given covariate information, using Super Learner (an ensemble machine learning algorithm also known as stacking) to learn the optimal combination of gradient boosting and random forest algorithms to maximize the area under the receiver operator curve. We evaluated variable importance (Shapley values) based on 3 levels: individual features, temporal windows, and clinical domains. We externally validated these findings using a holdout set of randomly selected study sites. RESULTS We were able to predict individual PASC diagnoses accurately (area under the curve 0.874). The individual features of the length of observation period, number of health care interactions during acute COVID-19, and viral lower respiratory infection were the most predictive of subsequent PASC diagnosis. Temporally, we found that baseline characteristics were the most predictive of future PASC diagnosis, compared with characteristics immediately before, during, or after acute COVID-19. We found that the clinical domains of health care use, demographics or anthropometry, and respiratory factors were the most predictive of PASC diagnosis. CONCLUSIONS The methods outlined here provide an open-source, applied example of using Super Learner to predict PASC status using electronic health record data, which can be replicated across a variety of settings. Across individual predictors and clinical domains, we consistently found that factors related to health care use were the strongest predictors of PASC diagnosis. This indicates that any observational studies using PASC diagnosis as a primary outcome must rigorously account for heterogeneous health care use. Our temporal findings support the hypothesis that clinicians may be able to accurately assess the risk of PASC in patients before acute COVID-19 diagnosis, which could improve early interventions and preventive care. Our findings also highlight the importance of respiratory characteristics in PASC risk assessment. INTERNATIONAL REGISTERED REPORT RR2-10.1101/2023.07.27.23293272


  • Open Access

    Practical considerations for specifying a super learner

    by Rachael V. Phillips, Mark J. van der Laan, Hana Lee, Susan Gruber

    Common tasks encountered in epidemiology, including disease incidence estimation and causal inference, rely on predictive modeling. Constructing a predictive model can be thought of as learning a prediction function, i.e., a function that takes as input covariate data and outputs a predicted value. Many strategies for learning these functions from data are available, from parametric regressions to machine learning algorithms. It can be challenging to choose an approach, as it is impossible to know in advance which one is the most suitable for a particular dataset and prediction task at hand. The super learner (SL) is an algorithm that alleviates concerns over selecting the one "right" strategy while providing the freedom to consider many of them, such as those recommended by collaborators, used in related research, or specified by subject-matter experts. It is an entirely pre-specified and data-adaptive strategy for predictive modeling. To ensure the SL is well-specified for learning the prediction function, the analyst does need to make a few important choices. In this Education Corner article, we provide step-by-step guidelines for making these choices, walking the reader through each of them and providing intuition along the way. In doing so, we aim to empower the analyst to tailor the SL specification to their prediction task, thereby ensuring their SL performs as well as possible. A flowchart provides a concise, easy-to-follow summary of key suggestions and heuristics, based on our accumulated experience, and guided by theory.


  • Top Cited Papers
    Open Access

    Practical considerations for specifying a super learner

    by Rachael V Phillips, Mark J van der Laan, Hana Lee, Susan Gruber

    Common tasks encountered in epidemiology, including disease incidence estimation and causal inference, rely on predictive modelling. Constructing a predictive model can be thought of as learning a prediction function (a function that takes as input covariate data and outputs a predicted value). Many strategies for learning prediction functions from data (learners) are available, from parametric regressions to machine learning algorithms. It can be challenging to choose a learner, as it is impossible to know in advance which one is the most suitable for a particular dataset and prediction task. The super learner (SL) is an algorithm that alleviates concerns over selecting the one ‘right’ learner by providing the freedom to consider many, such as those recommended by collaborators, used in related research or specified by subject-matter experts. Also known as stacking, SL is an entirely prespecified and flexible approach for predictive modelling. To ensure the SL is well specified for learning the desired prediction function, the analyst does need to make a few important choices. In this educational article, we provide step-by-step guidelines for making these decisions, walking the reader through each of them and providing intuition along the way. In doing so, we aim to empower the analyst to tailor the SL specification to their prediction task, thereby ensuring their SL performs as well as possible. A flowchart provides a concise, easy-to-follow summary of key suggestions and heuristics, based on our accumulated experience and guided by SL optimality theory.


  • Personalized online ensemble machine learning with applications for dynamic data streams

    by Ivana Malenica, Rachael V. Phillips, Antoine Chambaz, Alan E. Hubbard, Romain Pirracchio, Mark J. van der Laan

    In this work we introduce the personalized online super learner (POSL), an online personalizable ensemble machine learning algorithm for streaming data. POSL optimizes predictions with respect to baseline covariates, so personalization can vary from completely individualized, that is, optimization with respect to subject ID, to many individuals, that is, optimization with respect to common baseline covariates. As an online algorithm, POSL learns in real time. As a super learner, POSL is grounded in statistical optimality theory and can leverage a diversity of candidate algorithms, including online algorithms with different training and update times, fixed/offline algorithms that are not updated during POSL's fitting procedure, pooled algorithms that learn from many individuals' time series, and individualized algorithms that learn from within a single time series. POSL's ensembling of the candidates can depend on the amount of data collected, the stationarity of the time series, and the mutual characteristics of a group of time series. Depending on the underlying data‐generating process and the information available in the data, POSL is able to adapt to learning across samples, through time, or both. For a range of simulations that reflect realistic forecasting scenarios and in a medical application, we examine the performance of POSL relative to other current ensembling and online learning methods. We show that POSL is able to provide reliable predictions for both short and long time series, and it's able to adjust to changing data‐generating environments. We further cultivate POSL's practicality by extending it to settings where time series dynamically enter and exit.