Ensemble reliability and the signal‐to‐noise paradox in ECMWF subseasonal forecasts

Authors:
Open Access
Publisher website
Google Scholar
Abstract
Ensemble forecasts sometimes exhibit counterintuitive statistical properties, such that the correlation between ensemble means and observations () exceeds the correlation between ensemble means and individual members (), and thus models seem to underestimate the predictability of the real world. This behaviour has been interpreted as a “signal‐to‐noise paradox” (SNP), which is commonly diagnosed using the ratio of predictable components (). Here, we emphasise the links between ensemble‐size‐invariant estimates of RPC and other metrics of ensemble reliability and derive a general closed‐form expression for RPC in terms of , the spread–error ratio (SER), and the total variance ratio (VR). Physical constraints on the admissible solutions (i.e., real‐valued and non‐negative variances) provide a mechanism to identify statistically paradoxical sample estimates of RPC, , SER, and VR that correspond to combinations that are not possible without sampling uncertainty. We evaluate three large‐scale atmospheric circulation indices in subseasonal reforecasts from the European Centre for Medium‐Range Weather Forecasts (ECMWF) Integrated Forecasting System (IFS). Large‐ensemble North Atlantic Oscillation (NAO) forecasts evaluated over 80 start dates for the period 2001–2020 generally satisfy unbiased reliability criteria within our estimated sampling uncertainties, but still exhibit anomalously high RPC values at some subseasonal lead times. These lead times also coincide with paradoxical combinations of correlation and reliability metrics that are impossible in the large‐sample limit, indicating an important role for sampling uncertainties. Nevertheless, wintertime NAO indices averaged over days 16–45 exhibit more robust evidence for unreliability characterised by , suggesting that SNP‐like behaviour observed in daily data during the period 2001–2020 is not attributable solely to sampling artefacts. However, these results do not generalise to reduced ensemble size configurations of the same IFS model evaluated over 3120 start dates for the period 1959–2023. In these extended reforecasts, daily NAO indices are well calibrated and for all subseasonal lead times.

This publication has 67 references indexed in Scilit: