TS-HTFA: Advancing Time-Series Forecasting via Hierarchical Text-Free Alignment with Large Language Models
Open Access
Abstract
Given the significant potential of large language models (LLMs) in sequence modeling, emerging studies have begun applying them to time-series forecasting. Despite notable progress, existing methods still face two critical challenges: (1) their reliance on large amounts of paired text data, limiting the model applicability, and (2) a substantial modality gap between text and time series, leading to insufficient alignment and suboptimal performance. This paper introduces Hierarchical Text-Free Alignment (TS-HTFA) a novel method that leverages hierarchical alignment to fully exploit the representation capacity of LLMs for time-series analysis while eliminating the dependence on text data. Specifically, paired text data are replaced with adaptive virtual text based on QR decomposition word embeddings and learnable prompts. Furthermore, comprehensive cross-modal alignment is established at three levels: input, feature, and output, contributing to enhanced semantic symmetry between modalities. Extensive experiments on multiple time-series benchmarks demonstrate that TS-HTFA achieves state-of-the-art performance, significantly improving prediction accuracy and generalization.Keywords
Funding Information
- Inspur Cloud Information Technology Co., Ltd.
- Central Guidance for Local Science and Technology Development Fund of Shandong Province
This publication has 20 references indexed in Scilit:
- Residual-based Language Models are Free Boosters for Biomedical Imaging Tasks2024
- TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting MethodsProceedings of the VLDB Endowment, 2024
- Recommender Systems in the Era of Large Language Models (LLMs)IEEE Transactions on Knowledge and Data Engineering, 2024
- WFTNet: Exploiting Global and Local Periodicity in Long-Term Time Series Forecasting2024
- PromptCast: A New Prompt-Based Learning Paradigm for Time Series ForecastingIEEE Transactions on Knowledge and Data Engineering, 2023
- Deep learning models for price forecasting of financial time series: A review of recent advancements: 2020–2022WIREs Data Mining and Knowledge Discovery, 2023
- Are Transformers Effective for Time Series Forecasting?Proceedings of the AAAI Conference on Artificial Intelligence, 2023
- Multimodal Prompting with Missing Modalities for Visual Recognition2023
- DUMA: Dual Mask for Multivariate Time Series Anomaly DetectionIEEE Sensors Journal, 2022
- The M4 Competition: Results, findings, conclusion and way forwardInternational Journal of Forecasting, 2018