Optimising Time Series Forecasting Transformers to Linear Complexity
Authors:
Open Access
- 14 December 2025
- journal article
- Published by Institution of Engineering and Technology (IET) in Energy Internet
- Vol. 2 (4) , 275-286
Abstract
Accurate time series forecasting is a cornerstone of contemporary power and energy systems, supporting essential operational and planning functions including short‐term load prediction, electricity price forecasting and so on. Transformer‐based models achieve state‐of‐the‐art results by embedding series into tokens, but their quadratic time and space complexity limits the practical applications in real‐world scenarios. In this paper, we propose to chunk the tokens before feeding them to the encoder, in which attention calculations between tokens are limited to within the same chunk. Leveraging the frequency domain characteristics of the series, we employ locality sensitive hashing (LSH) techniques to ensure that series with similar frequency patterns are assigned to the same chunk. By fixing the chunk size, our method reduces the complexity of the transformer‐based model from to . Within each chunk, distilled features are concatenated to incorporate global information. Additionally, the running score update mechanism allows our framework to adapt the chunking strategy according to the data distribution. Extensive experiments demonstrate that, across multiple real‐world datasets and models, the proposed method maintains performance comparable to the original models while substantially reducing computational complexity and memory usage.Keywords
This publication has 21 references indexed in Scilit:
- Large Language Model-Aided Edge Learning in Distribution System State EstimationIEEE Internet of Things Journal, 2024
- Multi-Task Graph Adaptive Learning for Multivariate Electricity Price Short-Term Forecasting in Australia's National Electricity MarketIEEE Transactions on Power Systems, 2024
- Recurrent Neural Networks for Time Series Forecasting: Current status and future directionsInternational Journal of Forecasting, 2021
- The Performance of LSTM and BiLSTM in Forecasting Time Series2019
- The future of forecasting for renewable energyWires Energy and Environment, 2019
- Time series forecasting of petroleum production using deep LSTM recurrent networksNeurocomputing, 2019
- Probabilistic mid- and long-term electricity price forecastingRenewable and Sustainable Energy Reviews, 2018
- Probabilistic Forecasting of Wind Power Generation Using Extreme Learning MachineIEEE Transactions on Power Systems, 2013
- Time Series Analysis and Its Applications2000
- Exponential smoothing: The state of the artJournal of Forecasting, 1985