Sampling and misspecification errors in the estimation of observation‐error covariance matrices using observation‐minus‐background and observation‐minus‐analysis statistics
Open Access
- 18 May 2024
- journal article
- research article
- Published by Wiley in Quarterly Journal of the Royal Meteorological Society
- Vol. 150 (762) , 3052-3077
Abstract
Specification of the observation‐error covariance matrix for data assimilation systems affects the observation information content retained by the analysis, particularly for observations known to have correlated observation errors (e.g., geostationary satellite and Doppler radar data). A widely adopted approach for estimating observation‐error covariance matrices uses observation‐minus‐background and observation‐minus‐analysis residuals, which are routinely produced by most data assimilation systems. Although this approach is known to produce biased and noisy estimates, due to sampling and misspecification errors, there has been no systematic study of sampling errors with this approach to date. Furthermore, the eigenspectrum of the estimated observation‐error covariance matrix is known to influence the analysis information content and numerical convergence of variational assimilation schemes. In this work, we provide new theorems for the sampling error and eigenvalues of the estimated observation‐error covariance matrices with this approach. We also conduct numerical experiments to illustrate our theoretical results. We find that this method produces large sampling errors if the true observation‐error standard deviation is large, while the other error characteristics, including the true background‐error standard deviation and observation‐ and background‐error correlation length‐scales, have a relatively small effect. We also find that the smallest eigenvalues of the estimated matrices may be smaller or larger than the true eigenvalues, depending on the assumed and true observation‐ and background‐error statistics. These results may provide insights for practical applications: for example, in deciding on appropriate sample sizes and choosing parameters for matrix reconditioning techniques.Keywords
Funding Information
- Natural Environment Research Council
- Met Office
This publication has 48 references indexed in Scilit:
- Progress, challenges, and future steps in data assimilation for convection‐permitting numerical weather prediction: Report on the virtual meeting held on 10 and 12 November 2021Atmospheric Science Letters, 2022
- New bounds on the condition number of the Hessian of the preconditioned variational data assimilation problemNumerical Linear Algebra with Applications, 2021
- Comparing diagnosed observation uncertainties with independent estimates: A case study using aircraft‐based observations and a convection‐permitting data assimilation systemAtmospheric Science Letters, 2021
- The conditioning of least‐squares problems in variational data assimilationNumerical Linear Algebra with Applications, 2018
- Theoretical insight into diagnosing observation error correlations using observation‐minus‐background and observation‐minus‐analysis statisticsQuarterly Journal of the Royal Meteorological Society, 2015
- Error covariance estimation methods based on analysis residuals: theoretical foundation and convergence properties derived from simplified observation networksQuarterly Journal of the Royal Meteorological Society, 2015
- The benefits of correlated observation errors for small scalesQuarterly Journal of the Royal Meteorological Society, 2015
- Accounting for correlated error in the assimilation of high‐resolution sounder dataQuarterly Journal of the Royal Meteorological Society, 2013
- Estimating interchannel observation‐error correlations for IASI radiance data in the Met Office system†Quarterly Journal of the Royal Meteorological Society, 2013
- Correlated observation errors in data assimilationInternational Journal for Numerical Methods in Fluids, 2007