Learning All Dynamics: Traffic Forecasting via Locality-Aware Spatio-Temporal Joint Transformer
- 16 August 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 23 (12) , 23433-23446
Abstract
Forecasting traffic flow and speed in the urban is important for many applications, ranging from the intelligent navigation of map applications to congestion relief of city management systems. Therefore, mining the complex spatio-temporal correlations in the traffic data to accurately predict traffic is essential for the community. However, previous studies that combined the graph convolution network or self-attention mechanism with deep time series models (e.g., the recurrent neural network) can only capture spatial dependencies in each time slot and temporal dependencies in each sensor, ignoring the spatial and temporal correlations across different time slots and sensors. Besides, the state-of-the-art Transformer architecture used in previous methods is insensitive to local spatio-temporal contexts, which is hard to suit with traffic forecasting. To solve the above two issues, we propose a novel deep learning model for traffic forecasting, named Locality-aware spatio-temporal joint Transformer (Lastjormer), which elaborately designs a spatio-temporal joint attention in the Transformer architecture to capture all dynamic dependencies in the traffic data. Specifically, our model utilizes the dot-product self-attention on sensors across many time slots to extract correlations among them and introduces the linear and convolution self-attention mechanism to reduce the computation needs and incorporate local spatio-temporal information. Experiments on three real-world traffic datasets, England, METR-LA, and PEMS-BAY, demonstrate that our Lastjormer achieves state-of-the-art performances on a variety of challenging traffic forecasting benchmarks.Keywords
Funding Information
- National Natural Science Foundation of China (61872046)
- Beijing Natural Science Foundation (4212024)
- Science and Technology Plan Project of Inner Mongolia Autonomous Region (2019GG328)
This publication has 27 references indexed in Scilit:
- node2vec2016
- Scalable Algorithms for Data and Network AnalysisFoundations and Trends® in Theoretical Computer Science, 2016
- Effective Approaches to Attention-based Neural Machine Translation2015
- Data Mining2014
- Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation2014
- Travel-Time Prediction With Support Vector RegressionIEEE Transactions on Intelligent Transportation Systems, 2004
- Modeling and Forecasting Vehicular Traffic Flow as a Seasonal ARIMA Process: Theoretical Basis and Empirical ResultsJournal of Transportation Engineering, 2003
- Learning surface text patterns for a Question Answering system2001
- Long Short-Term MemoryNeural Computation, 1997
- Signal Processing of Speech1993