Parameter Efficient Deep Probabilistic Forecasting
Preprint
Abstract
Probabilistic time series forecasting is crucial in many application domains such as retail, ecommerce, finance, or biology. With the increasing availability of large volumes of data, a number of neural architectures have been proposed for this problem. In particular, Transformer-based methods achieve state-of-the-art performance on real-world benchmarks. However, these methods require a large number of parameters to be learned, which imposes high memory requirements on the computational resources for training such models. To address this problem, we introduce a novel Bidirectional Temporal Convolutional Network (BiTCN), which requires an order of magnitude less parameters than a common Transformer-based approach. Our model combines two Temporal Convolutional Networks (TCNs): the first network encodes future covariates of the time series, whereas the second network encodes past observations and covariates. We jointly estimate the parameters of an output distribution via these two networks. Experiments on four real-world datasets show that our method performs on par with four state-of-the-art probabilistic forecasting methods, including a Transformer-based approach and WaveNet, on two point metrics (sMAPE, NRMSE) as well as on a set of range metrics (quantile loss percentiles) in the majority of cases. Secondly, we demonstrate that our method requires significantly less parameters than Transformer-based methods, which means the model can be trained faster with significantly lower memory requirements, which as a consequence reduces the infrastructure cost for deploying these models.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- M5 accuracy competition: Results, findings, and conclusionsInternational Journal of Forecasting, 2022
- Principles and algorithms for forecasting groups of time series: Locality and globalityInternational Journal of Forecasting, 2021
- Recurrent Neural Networks for Time Series Forecasting: Current status and future directionsInternational Journal of Forecasting, 2021
- DeepAR: Probabilistic forecasting with autoregressive recurrent networksInternational Journal of Forecasting, 2020
- The M4 Competition: 100,000 time series and 61 forecasting methodsInternational Journal of Forecasting, 2020
- TADA: Trend Alignment with Dual-Attention Multi-task Recurrent Neural Networks for Sales Prediction2018
- Deep learning with long short-term memory networks for financial market predictionsEuropean Journal of Operational Research, 2018
- Forecasting at ScaleThe American Statistician, 2018
- Forecasting seasonals and trends by exponentially weighted moving averagesInternational Journal of Forecasting, 2004
- The theta model: a decomposition approach to forecastingInternational Journal of Forecasting, 2000