Scalable Spatiotemporal Graph Neural Networks
Preprint
Abstract
Neural forecasting of spatiotemporal time series drives both research and industrial innovation in several relevant application domains. Graph neural networks (GNNs) are often the core component of the forecasting architecture. However, in most spatiotemporal GNNs, the computational complexity scales up to a quadratic factor with the length of the sequence times the number of links in the graph, hence hindering the application of these models to large graphs and long temporal sequences. While methods to improve scalability have been proposed in the context of static graphs, few research efforts have been devoted to the spatiotemporal case. To fill this gap, we propose a scalable architecture that exploits an efficient encoding of both temporal and spatial dynamics. In particular, we use a randomized recurrent neural network to embed the history of the input time series into high-dimensional state representations encompassing multi-scale temporal dynamics. Such representations are then propagated along the spatial dimension using different powers of the graph adjacency matrix to generate node embeddings characterized by a rich pool of spatiotemporal features. The resulting node embeddings can be efficiently pre-computed in an unsupervised manner, before being fed to a feed-forward decoder that learns to map the multi-scale spatiotemporal representations to predictions. The training procedure can then be parallelized node-wise by sampling the node embeddings without breaking any dependency, thus enabling scalability to large networks. Empirical results on relevant datasets show that our approach achieves results competitive with the state of the art, while dramatically reducing the computational burden.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- TraverseNet: Unifying Space and Time in Message Passing for Traffic ForecastingIEEE Transactions on Neural Networks and Learning Systems, 2022
- Array programming with NumPyNature, 2020
- A gentle introduction to deep learning for graphsNeural Networks, 2020
- Reservoir Computing Approaches for Representation and Classification of Multivariate Time SeriesIEEE Transactions on Neural Networks and Learning Systems, 2020
- Design of deep echo state networksNeural Networks, 2018
- Deep reservoir computing: A critical experimental analysisNeurocomputing, 2017
- Reservoir computing approaches to recurrent neural network trainingComputer Science Review, 2009
- Correntropy: Properties and Applications in Non-Gaussian Signal ProcessingIEEE Transactions on Signal Processing, 2007
- Optimization and applications of echo state networks with leaky- integrator neuronsNeural Networks, 2007