Timestamp-Guided Knowledge Distillation for Robust Sensor-Based Time-Series Forecasting
Open Access
Abstract
Accurate time-series forecasting plays a vital role in sensor-driven applications such as energy monitoring, traffic flow prediction, and environmental sensing. While most existing approaches focus on extracting local patterns from historical observations, they often overlook the global temporal information embedded in timestamps. However, this information represents a valuable yet underutilized aspect of sensor-based data that can significantly enhance forecasting performance. In this paper, we propose a novel timestamp-guided knowledge distillation framework (TKDF), which integrates both historical and timestamp information through mutual learning between heterogeneous prediction branches to improve forecasting robustness. The framework comprises two complementary branches: a Backbone Model that captures local dependencies from historical sequences, and a Timestamp Mapper that learns global temporal patterns encoded in timestamp features. To enhance information transfer and reduce representational redundancy, a self-distillation mechanism is introduced within the Timestamp Mapper. Extensive experiments on multiple real-world sensor datasets—covering electricity consumption, traffic flow, and meteorological measurements—demonstrate that the TKDF consistently improves the performance of mainstream forecasting models.Keywords
Funding Information
- Major Scientific Project “Unveiling and Leading” of Ordos City (JBGS-2023-002)
This publication has 26 references indexed in Scilit:
- Deep Learning Models for Time Series Forecasting: A ReviewIEEE Access, 2024
- A multi-feature spatial–temporal fusion network for traffic flow predictionScientific Reports, 2024
- Deep knowledge distillation: A self-mutual learning framework for traffic predictionExpert Systems with Applications, 2024
- A Comparative Analysis of Early and Late Fusion for the Multimodal Two-Class ProblemIEEE Access, 2023
- Forecasting Time Series in Healthcare With Gaussian Processes and Dynamic Time Warping Based Subset SelectionIEEE Journal of Biomedical and Health Informatics, 2022
- Instance-based deep transfer learning with attention for stock movement predictionApplied Intelligence, 2022
- Deep Learning on Traffic Prediction: Methods, Analysis, and Future DirectionsIEEE Transactions on Intelligent Transportation Systems, 2021
- Ensemble learning for electricity consumption forecasting in office buildingsNeurocomputing, 2021
- Privacy-Preserving Traffic Flow Prediction: A Federated Learning ApproachIEEE Internet of Things Journal, 2020
- T-GCN: A Temporal Graph Convolutional Network for Traffic PredictionIEEE Transactions on Intelligent Transportation Systems, 2019