Mitigating Data Redundancy to Revitalize Transformer-Based Long-Term Time Series Forecasting System
Open Access
- 20 March 2026
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Intelligent Systems and Technology
- Vol. 17 (3) , 1-21
Abstract
Long-term time series forecasting (LTSF) is fundamental to various real-world applications, where Transformer-based models have become the dominant framework due to their ability to capture long-range dependencies. However, these models often experience overfitting due to data redundancy in rolling forecasting settings, limiting their generalization ability particularly evident in longer sequences with highly similar adjacent data. In this work, we introduce CLMFormer, a novel framework that mitigates redundancy through curriculum learning and a memory-driven decoder. Specifically, we progressively introduce Bernoulli noise to the training samples, which effectively breaks the high similarity between adjacent data points. This curriculum-driven noise introduction aids the memory-driven decoder by supplying more diverse and representative training data, enhancing the decoder’s ability to model seasonal tendencies and dependencies in the time series data. To further enhance forecasting accuracy, we introduce a memory-driven decoder. This component enables the model to capture seasonal tendencies and dependencies in the time series data and leverages temporal relationships to facilitate the forecasting process. Extensive experiments on six real-world LTSF benchmarks show that CLMFormer consistently improves Transformer-based models by up to 30%, demonstrating its effectiveness in long-horizon forecasting.Keywords
All Related Versions
Funding Information
- Australian Research Council (210101347)
- Australian Research Council (ARC) Discovery Early Career Researcher Award (DE190100626)
This publication has 29 references indexed in Scilit:
- TS-Fastformer: Fast Transformer for Time-series ForecastingACM Transactions on Intelligent Systems and Technology, 2024
- Deep Learning for Time Series Forecasting: Tutorial and Literature SurveyACM Computing Surveys, 2022
- Multivariate Time Series Forecasting With Dynamic Graph Neural ODEsIEEE Transactions on Knowledge and Data Engineering, 2022
- A survey on machine learning models for financial time series forecastingNeurocomputing, 2022
- Enhanced Bayesian Factorization With Variant Scale Partitioning for Multivariate Time Series AnalysisIEEE Transactions on Knowledge and Data Engineering, 2021
- Time2Graph+: Bridging Time Series and Graph Representation Learning via Multiple AttentionsIEEE Transactions on Knowledge and Data Engineering, 2021
- DeepAR: Probabilistic forecasting with autoregressive recurrent networksInternational Journal of Forecasting, 2020
- Adaptive Temporal-Frequency Network for Time-Series ForecastingIEEE Transactions on Knowledge and Data Engineering, 2020
- Transformer Based Memory Network for Sentiment Analysis of Web CommentsIEEE Access, 2019
- Exponential smoothing: The state of the artJournal of Forecasting, 1985