Melanie Schienle

Respiratory diseases cause considerable morbidity in autumn and winter and are a priority in public health monitoring. In Germany, they are subject to a number of surveillance systems, including both pathogen-specific and syndromic indicators. In this paper we present a collaborative multi-target and multi-model real-time forecasting system rolled out during the 2024/25 season, and discuss differences to earlier efforts carried out during the COVID-19 pandemic. We focus on general practitioner consultations for acute respiratory infections (ARI) and hospitalizations for severe acute respiratory infections (SARI), where a total of eight models were run to generate nowcasts and forecasts. In addition, we consider confirmed cases of seasonal influenza and respiratory syncytial virus (RSV) where one forecasting model each was available. As all indicators were subject to retrospective revisions, forecasting models were combined with a nowcasting step. For the SARI and ARI indicators, we combined the available models into an ensemble and conducted detailed quantitative evaluations. Nowcasts overall showed convincing performance, even though for some models Christmas break effects led to an upward bias in early January. Forecasts were overall well-calibrated and most models outperformed simple benchmark models. When taking into account age-stratified targets, most observed performance differences to a persistence baseline were statistically significant. Anticipating the peak timing and magnitude, however, proved to be challenging, with many models predicting too flat curves with a too early turnaround (e.g., in late January rather than mid-February for SARI). The combined ensemble forecast was among the best-performing approaches, but unlike in previous related projects did not consistently outperform individual models. We conclude by discussing learnings on the organization of collaborative forecasting projects in post-COVID-19 times. Author summary: Nowcasting and short-term forecasting are key tasks in epidemic modelling, and have received increasing attention from public health agencies in recent years. So-called Forecast Hubs have become a standard format to structure and coordinate modelling efforts. By collecting time-stamped predictions in standardized formats, they facilitate meaningful model comparisons and the generation of combined ensemble forecasts, which have often been found to be more robust. In this paper, we present the setup of a multi-model and multi-pathogen Forecasting Hub set up in Germany, along with detailed evaluation results for the 2024/25 season. Nowcasting models (i.e., models correcting recent incidence values for reporting delays) showed reliable performance. Forecasting models, on the other hand, mostly outperformed simple reference models, but struggled to anticipate peak timing and magnitude correctly. This paper adds to a growing body of evidence on the real-time performance of epidemic forecasts.

Predictive epidemic modeling can enhance situational awareness during emerging and seasonal outbreaks and has received increasing interest in recent years. A common distinction is between nowcasting, which corrects recent incidence data for reporting delays, and forecasting, which predicts future trends. This paper presents an integrated system for nowcasting and multi-model short-term forecasting of hospitalizations from severe acute respiratory infections (SARI) in Germany (November 2023–September 2024). We propose a modular approach combining a statistical nowcasting model with various data-driven forecasting methods, including a time series model, a gradient boosting approach, and a neural network. These are combined into an ensemble approach, which achieves the best average performance. The resulting forecasts are overall well-calibrated up to four weeks ahead, but struggled to capture the unusual double peak which occurred during the test season. While the presented analysis is retrospective, it serves as a blueprint for a collaborative real-time forecasting platform for respiratory diseases in Germany (the RESPINOW Hub ). We conclude with an outlook on this system, which was launched in the fall of 2024 and covers a broader range of data sources and modeling approaches.

Real-time surveillance is a crucial element in the response to infectious disease outbreaks. However, the interpretation of incidence data is often hampered by delays occurring at various stages of data gathering and reporting. As a result, recent values are biased downward, which obscures current trends. Statistical nowcasting techniques can be employed to correct these biases, allowing for accurate characterization of recent developments and thus enhancing situational awareness. In this paper, we present a preregistered real-time assessment of eight nowcasting approaches, applied by independent research teams to German 7-day hospitalization incidences. This indicator played an important role in the management of the pandemic in Germany and was linked to levels of non-pharmaceutical interventions via certain thresholds. Due to its definition, in which hospitalization counts are aggregated by the date of case report rather than admission, German hospitalization incidences are particularly affected by delays and can take several weeks or months to fully stabilize. For this study, all methods were applied from 22 November 2021 to 29 April 2022, with probabilistic nowcasts produced each day for the current and 28 preceding days. Nowcasts at the national, state, and age-group levels were collected in the form of quantiles in a public repository and displayed in a dashboard. Moreover, a mean and a median ensemble nowcast were generated. We find that overall, the compared methods were able to remove a large part of the biases introduced by delays. Most participating teams underestimated the importance of very long delays, though, resulting in nowcasts with a slight downward bias. The accompanying uncertainty intervals were also too narrow for almost all methods. Averaged over all nowcast horizons, the best performance was achieved by a model using case incidences as a covariate and taking into account longer delays than the other approaches. For the most recent days, which are often considered the most relevant in practice, a mean ensemble of the submitted nowcasts performed best. We conclude by providing some lessons learned on the definition of nowcasting targets and practical challenges.

Background: During the COVID-19 pandemic there has been a strong interest in forecasts of the short-term development of epidemiological indicators to inform decision makers. In this study we evaluate probabilistic real-time predictions of confirmed cases and deaths from COVID-19 in Germany and Poland for the period from January through April 2021. Methods: We evaluate probabilistic real-time predictions of confirmed cases and deaths from COVID-19 in Germany and Poland. These were issued by 15 different forecasting models, run by independent research teams. Moreover, we study the performance of combined ensemble forecasts. Evaluation of probabilistic forecasts is based on proper scoring rules, along with interval coverage proportions to assess calibration. The presented work is part of a pre-registered evaluation study. Results: We find that many, though not all, models outperform a simple baseline model up to four weeks ahead for the considered targets. Ensemble methods show very good relative performance. The addressed time period is characterized by rather stable non-pharmaceutical interventions in both countries, making short-term predictions more straightforward than in previous periods. However, major trend changes in reported cases, like the rebound in cases due to the rise of the B.1.1.7 (Alpha) variant in March 2021, prove challenging to predict. Conclusions: Multi-model approaches can help to improve the performance of epidemiological forecasts. However, while death numbers can be predicted with some success based on current case and hospitalization data, predictability of case numbers remains low beyond quite short time horizons. Additional data sources including sequencing and mobility data, which were not extensively used in the present study, may help to improve performance.

Background: During the COVID-19 pandemic there has been a strong interest in forecasts of the short-term development of epidemiological indicators to inform decision makers. In this study we evaluate probabilistic real-time predictions of confirmed cases and deaths from COVID-19 in Germany and Poland for the period from January through April 2021. Methods: We evaluate probabilistic real-time predictions of confirmed cases and deaths from COVID-19 in Germany and Poland. These were issued by 15 different forecasting models, run by independent research teams. Moreover, we study the performance of combined ensemble forecasts. Evaluation of probabilistic forecasts is based on proper scoring rules, along with interval coverage proportions to assess forecast calibration. The presented work is part of a pre-registered evaluation study and covers the period from January through April 2021. Results: We find that many, though not all, models outperform a simple baseline model up to four weeks ahead for the considered targets. Ensemble methods (i.e., combinations of different available forecasts) show very good relative performance. The addressed time period is characterized by rather stable non-pharmaceutical interventions in both countries, making short-term predictions more straightforward than in previous periods. However, major trend changes in reported cases, like the rebound in cases due to the rise of the B.1.1.7 (alpha) variant in March 2021, prove challenging to predict. Conclusions: Multi-model approaches can help to improve the performance of epidemiological forecasts. However, while death numbers can be predicted with some success based on current case and hospitalization data, predictability of case numbers remains low beyond quite short time horizons. Additional data sources including sequencing and mobility data, which were not extensively used in the present study, may help to improve performance. Plain language summary: The goal of this study is to assess the quality of forecasts of weekly case and death numbers of COVID-19 in Germany and Poland during the period of January through April 2021. We focus on real-time forecasts at time horizons of one and two weeks ahead created by fourteen independent teams. Forecasts are systematically evaluated taking uncertainty ranges of predictions into account. We find that combining different forecasts into ensembles can improve the quality of predictions, but especially case numbers proved very challenging to predict beyond quite short time windows. Additional data sources, in particular genetic sequencing data, may help to improve forecasts in the future.

Disease modelling has had considerable policy impact during the ongoing COVID-19 pandemic, and it is increasingly acknowledged that combining multiple models can improve the reliability of outputs. Here we report insights from ten weeks of collaborative short-term forecasting of COVID-19 in Germany and Poland (12 October–19 December 2020). The study period covers the onset of the second wave in both countries, with tightening non-pharmaceutical interventions (NPIs) and subsequently a decay (Poland) or plateau and renewed increase (Germany) in reported cases. Thirteen independent teams provided probabilistic real-time forecasts of COVID-19 cases and deaths. These were reported for lead times of one to four weeks, with evaluation focused on one- and two-week horizons, which are less affected by changing NPIs. Heterogeneity between forecasts was considerable both in terms of point predictions and forecast spread. Ensemble forecasts showed good relative performance, in particular in terms of coverage, but did not clearly dominate single-model predictions. The study was preregistered and will be followed up in future phases of the pandemic.

We use official data for all 16 federal German states to study the causal effect of a flat 1000 Euro state-dependent university tuition fee on the enrollment behavior of students during the years 2006-2014. In particular, we show how the variation in the introduction scheme across states and times can be exploited to identify the federal average causal effect of tuition fees by controlling for a large amount of potentially influencing attributes for state heterogeneity. We suggest a stability post-double selection methodology to robustly determine the causal effect across types in the transparently modeled unknown response components. The proposed stability resampling scheme in the two LASSO selection steps efficiently mitigates the risk of model underspecification and thus biased effects when the tuition fee policy decision also depends on relevant variables for the state enrollment rates. Correct inference for the full cross-section state population in the sample requires adequate design -- rather than sampling-based standard errors. With the data-driven model selection and explicit control for spatial cross-effects we detect that tuition fees induce substantial migration effects where the mobility occurs both from fee but also from non-fee states suggesting also a general movement for quality. Overall, we find a significant negative impact of up to 4.5 percentage points of fees on student enrollment. This is in contrast to plain one-step LASSO or previous empirical studies with full fixed effects linear panel regressions which generally underestimate the size and get an only insignificant effect.