Finite Samples and Uncertainty Estimates for Skill Measures for Seasonal Prediction
Author:
Open Access
- 1 August 2009
- journal article
- Published by American Meteorological Society in Monthly Weather Review
- Vol. 137 (8) , 2622-2631
Abstract
The expected value for various measures of skill for seasonal climate predictions is determined by the signal-to-noise ratio. The expected value, however, is only realized for long verification time series. In practice, the verifications for specific seasons—for example, forecasts for the December–February seasonal mean—seldom exceed a sample size of 30. The estimates of skill measure based on small verification time series, because of sampling errors, can have large departures from their expected value. An analysis of spread in the estimates of skill measures with the length of verification time series and for different signal-to-noise ratios is made. The analysis is based on the Monte Carlo approach and skill measures for deterministic, categorical, and probabilistic forecasts are considered. It is shown that the behavior of spread for various skill measures can be very different and it is not always the largest for the small values of signal-to-noise ratios.Keywords
This publication has 13 references indexed in Scilit:
- A Predictability Measure Applied to Seasonal Predictions of the Arctic OscillationJournal of Climate, 2007
- On the Interpretation and Utility of Skill Information for Seasonal Climate PredictionsMonthly Weather Review, 2007
- Measuring the potential utility of seasonal climate predictionsGeophysical Research Letters, 2004
- Seasonal Predictions, Probabilistic Verifications, and Ensemble SizeJournal of Climate, 2001
- Changes of Probability Associated with El NiñoJournal of Climate, 2000
- Changes in the Spread of the Variability of the Seasonal Mean Atmospheric States Associated with ENSOJournal of Climate, 2000
- Analysis of a Conceptual Model of Seasonal Climate Variability and Implications for Seasonal PredictionBulletin of the American Meteorological Society, 2000
- Prospects and Limitations of Seasonal Atmospheric GCM PredictionsBulletin of the American Meteorological Society, 1995
- Correspondence among the Correlation, RMSE, and Heidke Forecast Verification Measures; Refinement of the Heidke ScoreWeather and Forecasting, 1992
- Global and Regional Scale Precipitation Patterns Associated with the El Niño/Southern OscillationMonthly Weather Review, 1987