MeteoFormer: The Accuracy–Faithfulness Trade-Off of Built-In Attention in Meteorological Forecasting

Authors:
Open Access
Publisher website
Google Scholar
Abstract
Built-in attention is often presented as a free explanation of what a forecasting transformer relies on, but the claim is rarely tested. We test it on MeteoFormer, a patch-based transformer with single-head variable and temporal attention that forecasts four weather variables 24 h ahead from 96 h of hourly input on the Jena and Beijing PRSA datasets. Against ten baselines trained under one protocol with five seeds, MeteoFormer has the lowest mean absolute error on Jena (3.025 against 3.107 for Crossformer, Diebold–Mariano p < 0.05 against every baseline after Holm correction) and is level with Crossformer on Beijing (4.151 against 4.138, p = 0.72), while Crossformer has the lower squared error, a difference traced to humidity. The model depends heavily on the latest observations: losing the last 16 h raises its Jena error by 73% and drops it from first to sixth place. The explanation claim fails. The fidelity of the variable attention does not differ from zero (0.012 on Jena, 0.046 on Beijing), far below SHAP on the same model; the temporal attention is almost uniform although the forecast depends mainly on the last 16 h; and the configuration selected for accuracy has the least faithful attention (0.012 against 0.181 with instance normalization, p = 0.001), but the more faithful configuration is less accurate (Jena MAE 3.195 against 3.025) and its attention remains below SHAP.
Funding Information
  • “Scientific Research of Young Scientists within the Grant of the Chairman of the Board and Rector of Korkyt Ata Kyzylorda University”

This publication has 70 references indexed in Scilit: