Alyssa Bilinski

Several key metrics in public health convey the probability that a primary event will lead to a more serious secondary event in the future. These “severity rates” can change over the course of an epidemic in response to shifting conditions, like new therapeutics, variants, or public health interventions. In practice, time-varying parameters, such as the case-fatality rate, are typically estimated from aggregate count data. Prior work has demonstrated that commonly-used ratio-based estimators can be highly biased, motivating the development of new methods. In this paper we develop an adaptive deconvolution approach based on approximating a Poisson-binomial model for secondary events, and we regularize the maximum likelihood solution in this model with a trend filtering penalty to produce smooth but locally adaptive estimates of severity rates over time. This enables us to compute severity rates both retrospectively and in real time. Experiments based on COVID-19 death and hospitalization data show that our deconvolution estimator is generally more accurate than the standard ratio-based methods and displays reasonable robustness to model misspecification.

Objective: The population-level quality-of-life impact of nonfatal health outcomes, including prolonged hospitalizations and long COVID, remains poorly understood. In this study, we estimate the cumulative quality-adjusted life-year (QALY) loss attributable to fatal and nonfatal COVID-19 outcomes in the United States and compare that burden with that of other diseases. Methods: We developed a probabilistic model that accounts for the loss in population health due to COVID-19 symptoms, hospitalizations, long COVID, and deaths. The model was applied to weekly county-level data from July 2020 to December 2022. We used appropriate probability distributions to describe uncertainty in parameter values and used 1,000 repeated samples from these distributions to estimate the expected QALY loss and the 95% uncertainty interval (UI). Results: We estimate the cumulative QALY loss as 9,242,000 (95% UI: 7,808,000–11,311,000) life-years. Following deaths, long COVID was the second most significant contributor to QALY loss, with an estimated annual QALY loss of approximately 865,000 (95% UI: 355,000–1,591,000). This burden is comparable to the total annual disability-adjusted life-years (DALYs) associated with liver cancer: 581,000 DALYs (95% UI: 549,000–607,000). Although symptomatic infections contributed less to overall QALY loss, their burden was comparable to that of HIV/AIDS (377,000 DALYs, 95% UI: 309,000–465,000). Conclusion: The quality-of-life burden of nonfatal COVID-19 outcomes, particularly long COVID, is on the same scale as that of major chronic and infectious diseases. As COVID-19 mortality declines and the pandemic transitions to an endemic phase, it is essential to recognize and address the long-term effects of nonfatal outcomes as public health priorities.

Difference‐in‐differences is a popular method for observational health policy evaluation. It relies on a causal assumption that in the absence of intervention, treatment groups' outcomes would have evolved in parallel to those of comparison groups. Researchers frequently look for parallel trends in the pre‐intervention period to bolster confidence in this assumption. The popular “parallel trends test” evaluates a null hypothesis of parallel trends and, failing to find evidence against the null, concludes that the assumption holds. This tightly controls the probability of falsely concluding that trends are not parallel, but may have low power to detect non‐parallel trends. When used as a screening step, it can also introduce bias in treatment effect estimates. We propose a non‐inferiority/equivalence approach that tightly controls the probability of missing large violations of parallel trends, measured on the scale of the treatment effect. Our framework nests several common use cases, including linear trend tests and event studies. We show that our approach may induce no or minimal bias when used as a screening step under commonly assumed error structures and, absent violations, can offer a higher‐power alternative to testing treatment effects in more flexible models. We illustrate our ideas by reconsidering a study of the impact of the Affordable Care Act's dependent coverage provision.

Introduction: During the COVID-19 pandemic, many communities across the United States experienced surges in hospitalizations, which strained the local hospital capacity. Some risk metrics, such as the Center for Disease Control and Prevention’s (CDC’s) Community Levels, were developed to predict the impact of COVID-19 on the community-level health care system based on routine surveillance data. However, they had limited utility as they were not routinely updated based on accumulating data and were not directly linked to specific outcomes, such as surges in COVID-19 hospitalizations beyond local capacities. Methods: In this article, we evaluated decision tree classifiers developed in real time to predict surges in local hospitalizations due to COVID-19 between July 2020 and November 2022. These classifiers would have provided visually intuitive and interpretable decision rules and, by being updated weekly, would have responded to changes in the epidemic. We compared the performance of these classifiers with that of logistic regression and neural network models using various metrics, including the area under the receiver-operating characteristic curve (auROC) and the area under the precision-recall curve (auPRC). Results: Decision tree classifiers achieved an auROC of > 80 % for most pandemic weeks and outperformed the CDC’s Community Levels in predicting high hospital occupancy. The auPRC, sensitivity, and specificity of the classifiers varied more substantially over time (between 20 % and 100 % ) and in sync with pandemic waves. Decision tree classifiers demonstrated similar performance compared with logistic regression and neural network models while presenting more interpretable classification rules. Conclusions: Using routinely collected hospital surveillance data, decision tree classifiers can be adaptively updated to predict surges in local hospitalizations. However, the sensitivity and specificity of these classifiers could change markedly during different pandemic waves. Highlights: A major concern during the COVID-19 pandemic was the risk of exceeding local health care capacity due to COVID-19-related hospitalizations. To assess this risk and inform mitigating strategies, several risk assessment tools were developed during the pandemic. Many of these tools, however, did not predict local outcomes, were not updated as the pandemic progressed, and/or were not interpretable by decision makers. We propose an adaptive framework of decision tree classifiers to predict whether COVID-19–related hospital occupancy would exceed a given capacity threshold. These classifiers demonstrated reasonable and stable prediction performance over time. However, their sensitivity and specificity may change substantially over the course of pandemic waves.

During the COVID-19 pandemic, estimating the total deaths averted by vaccination was of great public health interest. Instead of estimating total deaths averted by vaccination among both vaccinated and unvaccinated individuals, some studies empirically estimated only “directly averted” deaths among vaccinated individuals, typically suggesting that vaccines prevented more deaths among unvaccinated and vaccinated individuals than directly among vaccinated individuals only, due to the indirect effect. Here, we define the causal estimand to quantify outcomes “directly averted” by vaccination—that is, the impact of vaccination for vaccinated individuals, holding vaccination coverage fixed—for vaccination at multiple time points, which is a lower bound on the total outcomes averted when the indirect effect is non-negative. We develop an unbiased estimator for the causal estimand in a one-stage randomized controlled trial and explore the bias of a popular “hazard difference” estimator frequently used in empirical studies. We show that even in a randomized controlled trial, the hazard difference estimator is biased if vaccination has a non-null effect, as it fails to incorporate the greater depletion of susceptibles among the unvaccinated individuals. In simulations, the overestimation is small for averted deaths when the infection-fatality rate is low, as for many important pathogens. However, the overestimation can be large for averted infections given a high basic reproduction number and a high vaccine efficacy against infection. Additionally, we define and compare estimand and estimators for avertible outcomes (i.e., outcomes that could have been averted by vaccination, but were not due to failure to vaccinate). Future studies can explore the identifiability of the causal estimand in observational settings.

Difference-in-differences (DiD) approach is one of the most widely used approaches for evaluating policy effects. However, traditional DiD methods may not recover the population-level average treatment effect on the treated (ATT) in the absence of population-level panel data, particularly when the composition of units in the treatment group changes over time. In this work, we address the following two challenges when applying DiD methods with repeated cross-sectional (RCS) survey data: (1) heterogeneous compositions of study samples across different time points, and (2) availability of data for only a sample of the population. We introduce a policy-relevant target estimand and establish its identification conditions. We then propose a new weighting approach that incorporates both estimated propensity scores and given survey weights. We establish the theoretical properties of the proposed method and examine its finite-sample performance through simulations. Finally, we apply our proposed method to a real-world data application, estimating the effect of a beverage tax on adolescent soda consumption in Philadelphia.

Difference‐in‐differences (DiD) is a popular observational causal inference method in health policy, employed to evaluate the real‐world impact of policies and programs. To estimate treatment effects, DiD relies on a “parallel trends assumption” that treatment and comparison groups would have had parallel trajectories on average in the absence of an intervention. Recent years have seen both growing use of DiD in health policy and medicine and rapid advancements in DiD methods. To support DiD implementation in these fields, this paper reviews and synthesizes best practices and recent innovations. We provide recommendations to practitioners in four areas: (1) assessing causal assumptions; (2) adjusting for covariates and other approaches to relax causal assumptions; (3) accounting for staggered treatment timing; and (4) conducting robust inference, especially when normal‐based clustered standard errors are inappropriate. For each, we explain challenges and common pitfalls in traditional DiD and recommend methods to address these. We explore current treatment of these topics through a focused literature review of medical DiD studies.