Thomas J. Hladish

We evaluate approaches to vaccine distribution using an agent-based model of human activity and COVID-19 transmission calibrated to detailed trends in cases, hospitalizations, deaths, seroprevalence, and vaccine breakthrough infections in Florida, USA. We compare the incremental effectiveness for four different distribution strategies at four different levels of vaccine supply, starting in late 2020 through early 2022. Our analysis indicates that the best strategy to reduce severe outcomes would be to actively target high disease-risk individuals. This was true in every scenario, although the advantage was greatest for the intermediate vaccine availability assumptions and relatively modest compared to a simple mass vaccination approach under high vaccine availability. Ring vaccination, while generally the most effective strategy for reducing infections, ultimately proved least effective at preventing deaths. We also consider using age group as a practical surrogate measure for actual disease-risk targeting; this approach also outperforms both simple mass distribution and ring vaccination. We find that quantitative effectiveness of a strategy depends on whether effectiveness is assessed after the alpha, delta, or omicron wave. However, these differences in absolute benefit for the strategies do not change the ranking of their performance at preventing severe outcomes across vaccine availability assumptions. We use our agent-based model of SARS-CoV-2 transmission to evaluate alternative vaccine distribution strategies over a range of vaccine supply scenarios. We find that strategies targeting transmission (e.g., ring vaccination) perform best in preventing infections, but targeting disease risk prevents more instances of severe outcomes. Specifically, strategies based on age, or age and comorbidities—which do not require contact tracing—resulted in the fewest hospitalizations in our model. These strategy rankings held true across all vaccine supply scenarios and were robust to the introduction of SARS-CoV-2 variants. While the quantitative results cannot be directly applied to other settings (as we used a synthetic population calibrated to the State of Florida), the rankings of strategies should be more generalizable.

Our ability to forecast epidemics more than a few weeks into the future is constrained by the complexity of disease systems, our limited ability to measure the current state of an epidemic, and uncertainties in how human action will affect transmission. Realistic longer-term projections (spanning more than a few weeks) may, however, be possible under defined scenarios that specify the future state of critical epidemic drivers, with the additional benefit that such scenarios can be used to anticipate the comparative effect of control measures. Since December 2020, the U.S. COVID-19 Scenario Modeling Hub (SMH) has convened multiple modeling teams to make 6-month ahead projections of the number of SARS-CoV-2 cases, hospitalizations and deaths. The SMH released nearly 1.8 million national and state-level projections between February 2021 and November 2022. SMH performance varied widely as a function of both scenario validity and model calibration. Scenario assumptions were periodically invalidated by the arrival of unanticipated SARS-CoV-2 variants, but SMH still provided projections on average 22 weeks before changes in assumptions (such as virus transmissibility) invalidated scenarios and their corresponding projections. During these periods, before emergence of a novel variant, a linear opinion pool ensemble of contributed models was consistently more reliable than any single model, and projection interval coverage was near target levels for the most plausible scenarios (e.g., 79% coverage for 95% projection interval). SMH projections were used operationally to guide planning and policy at different stages of the pandemic, illustrating the value of the hub approach for long-term scenario projections.

Policymakers must make management decisions despite incomplete knowledge and conflicting model projections. Little guidance exists for the rapid, representative, and unbiased collection of policy-relevant scientific input from independent modeling teams. Integrating approaches from decision analysis, expert judgment, and model aggregation, we convened multiple modeling teams to evaluate COVID-19 reopening strategies for a mid-sized United States county early in the pandemic. Projections from seventeen distinct models were inconsistent in magnitude but highly consistent in ranking interventions. The 6-mo-ahead aggregate projections were well in line with observed outbreaks in mid-sized US counties. The aggregate results showed that up to half the population could be infected with full workplace reopening, while workplace restrictions reduced median cumulative infections by 82%. Rankings of interventions were consistent across public health objectives, but there was a strong trade-off between public health outcomes and duration of workplace closures, and no win-win intermediate reopening strategies were identified. Between-model variation was high; the aggregate results thus provide valuable risk quantification for decision making. This approach can be applied to the evaluation of management interventions in any setting where models are used to inform decision making. This case study demonstrated the utility of our approach and was one of several multimodel efforts that laid the groundwork for the COVID-19 Scenario Modeling Hub, which has provided multiple rounds of real-time scenario projections for situational awareness and decision making to the Centers for Disease Control and Prevention since December 2020.

We evaluate approaches to vaccine distribution using an agent-based model of human activity and COVID-19 transmission calibrated to detailed trends in cases, hospitalizations, deaths, seroprevalence, and vaccine breakthrough infections in Florida, USA. We compare the incremental effectiveness for four different distribution strategies at four different levels of vaccine availability, reflecting different income settings’ historical COVID-19 vaccine distribution. Our analysis indicates that the best strategy to reduce severe outcomes is to actively target high disease-risk individuals. This was true in every scenario, although the advantage was greatest for the middle-income-country availability assumptions, and relatively modest compared to a simple mass vaccination approach for rapid, high levels of vaccine availability. Ring vaccination, while generally the most effective strategy for reducing infections, ultimately proved least effective at preventing deaths. We also consider using age group as a practical, surrogate measure for actual disease-risk targeting; this approach still outperforms both simple mass distribution and ring vaccination. We also find that the magnitude of strategy effectiveness depends on when assessment occurs ( e.g ., after delta vs. after omicron variants). However, these differences in absolute benefit for the strategies do not change the ranking of their performance at preventing severe outcomes across vaccine availability assumptions.

Background Decision-makers impose COVID-19 mitigations based on public health indicators such as reported cases, which are sensitive to fluctuations in supply and demand for diagnostic testing, and hospital admissions, which lag infections by up to two weeks. Imposing mitigations too early has unnecessary economic costs, while imposing too late leads to uncontrolled epidemics with unnecessary cases and deaths. Sentinel surveillance of recently-symptomatic individuals in outpatient testing sites may overcome biases and lags in conventional indicators, but the minimal outpatient sentinel surveillance system needed for reliable trend estimation remains unknown. Methods We used a stochastic, compartmental transmission model to evaluate the performance of various surveillance indicators at reliably triggering an alarm in response to, but not before, a step increase in transmission of SARS-CoV-2. The surveillance indicators included hospital admissions, hospital occupancy, and sentinel cases with varying levels of sampling effort capturing 5, 10, 20, 50 or 100% of incident mild cases. We tested 3 levels of transmission increase, 3 population sizes, and condition of either simultaneous transmission increase, or lagged increase in older population. We compared the indicators’ performance at triggering alarm soon after, but not prior, to the transmission increase. Results Compared to surveillance based on hospital admissions, outpatient sentinel surveillance that captured at least 20% of incident mild cases could trigger alarm 2 to 5 days earlier for a mild increase in transmission and 6 days earlier for moderate or strong increase. Sentinel surveillance triggered fewer false alarms and averted more deaths per day spent in mitigation. When transmission increase in older populations lagged increase in younger populations by 14 days, sentinel surveillance extended its lead time over hospital admissions by an additional 2 days. Conclusions Sentinel surveillance of mild symptomatic cases can provide more timely and reliable information on changes in transmission to inform decision-makers in an epidemic like COVID-19.

In this report, we use a detailed simulation model to assess and project the COVID-19 epidemic in Florida. The model is a data-driven, stochastic, discrete-time, agent based model with an explicit representation of people and places. Using the model, we find that the omicron variant wave in Florida is likely to cause many more infections than occurred during the delta variant wave. Due to testing limitations and often mild symptoms, however, we anticipate that omicron infections will be underreported compared to delta. We project that reported cases of COVID-19 will continue to grow significantly and peak in early January 2022, and that the number of reported COVID-19 deaths due to omicron may be 1/3 of the total caused by the delta wave.

Complex models are often fitted to data using simulation-based calibration, a computationally challenging process. Several calibration methods to improve computational efficiency have been developed with no consensus on which methods perform best. We did a simulation study comparing the performance of 5 methods that differed in their Goodness-of-Fit (GOF) metrics and parameter search strategies. Posterior densities for two parameters of a simple Susceptible-Infectious-Recovered epidemic model were obtained for each calibration method under two scenarios. Scenario 1 (S1) allowed 60K model runs and provided two target statistics, whereas scenario 2 (S2) allowed 75K model runs and provided three target statistics. For both scenarios, we obtained reference posteriors against which we compare all other methods by running Rejection ABC for 5M parameter combinations, retaining the 0.1% best. We assessed performance by applying a 2D-grid to all posterior densities and quantifying the percentage overlap with the reference posterior. We considered basic and adaptive sampling calibration methods. Of the basic calibration methods, Bayesian calibration (Bc) Sampling Importance Resampling (S1: 34.8%, S2: 39.8%) outperformed Rejection Approximate Bayesian Computation (ABC) (S1: 2.3%, S2: 1.8%). Among the adaptive sampling methods, Bc Incremental Mixture Importance Sampling (S1: 72.7%, S2: 85.5%) outperformed sequential Monte Carlo ABC (AbcSmc) (S1: 53.9%, S2: 72.9%) and Sequential ABC (S1: 21.6%, S2: 62.7%). Basic methods led to sub-optimal calibration results. Methods using the surrogate Likelihood as a GOF outperformed methods using a distance measure. Adaptive sampling methods were more efficient compared to their basic counterparts and resulted in accurate posterior distributions. BcIMIS was the best performing method. When three rather than two target statistics were available, the difference in performance between the adaptive sampling methods was less pronounced. Although BcIMIS outperforms the other methods, limitations related to the target statistics and available computing infrastructure may warrant the choice of an alternative method. Author summary As mathematical models become more realistic, they tend to become more complex. Calibration, the process of tuning a model to better reproduce empirical data, can become dramatically more computationally intensive as model complexity increases. Researchers have responded by developing a range of more efficient, adaptive sampling calibration methods. However, the relative performance of these calibration methods remains unclear. To this end, we quantified the performance of five commonly used calibration methods. We found that adaptive sampling methods were more efficient compared to their basic counterparts and resulted in more accurate posterior distributions. We identified the best performing method, but caution that limitations related to the target statistics and available computing infrastructure may warrant the choice of one of the alternatives. Finally, we provide the code used to apply the calibration methods in our study as a primer to facilitate their application.

Viruses transmitted by Aedes mosquitoes, such as dengue, Zika, and chikungunya, have expanding ranges and seem unabated by current vector control programs. Effective control of these pathogens likely requires integrated approaches. We evaluated dengue management options in an endemic setting that combine novel vector control and vaccination using an agent-based model for Yucatán, Mexico, fit to 37 y of data. Our intervention models are informed by targeted indoor residual spraying (TIRS) experiments; trial outcomes and World Health Organization (WHO) testing guidance for the only licensed dengue vaccine, CYD-TDV; and preliminary results for in-development vaccines. We evaluated several implementation options, including varying coverage levels; staggered introductions; and a one-time, large-scale vaccination campaign. We found that CYD-TDV and TIRS interfere: while the combination outperforms either alone, performance is lower than estimated from their separate benefits. The conventional model hypothesized for in-development vaccines, however, performs synergistically with TIRS, amplifying effectiveness well beyond their independent impacts. If the preliminary performance by either of the in-development vaccines is upheld, a one-time, large-scale campaign followed by routine vaccination alongside aggressive new vector control could enable short-term elimination, with nearly all cases avoided for a decade despite continuous dengue reintroductions. If elimination is impracticable due to resource limitations, less ambitious implementations of this combination still produce amplified, longer-lasting effectiveness over single-approach interventions. Significance Using realistic simulation of dengue and plausible control programs in the state of Yucatán, Mexico, we show that vaccination programs can have amplifying or interfering dynamic effects when combined with expanded vector control effort. These results should inform planning in regions considering an integrated dengue control program, particularly if using CYD-TDV, which we found to underperform when combined with increasing vector control.