Temporal Overdrive Recurrent Neural Network
Preprint
Abstract
In this work we present a novel recurrent neural network architecture designed to model systems characterized by multiple characteristic timescales in their dynamics. The proposed network is composed by several recurrent groups of neurons that are trained to separately adapt to each timescale, in order to improve the system identification process. We test our framework on time series prediction tasks and we show some promising, preliminary results achieved on synthetic data. To evaluate the capabilities of our network, we compare the performance with several state-of-the-art recurrent architectures.Keywords
All Related Versions
This publication has 14 references indexed in Scilit:
- Training Echo State Networks with Regularization Through Dimensionality ReductionCognitive Computation, 2017
- LSTM: A Search Space OdysseyIEEE Transactions on Neural Networks and Learning Systems, 2016
- Prediction of telephone calls load using Echo State Network with exogenous variablesNeural Networks, 2015
- Forecasting time series with multiple seasonal patternsEuropean Journal of Operational Research, 2008
- Band-pass Reservoir Computing2008
- Optimization and applications of echo state networks with leaky- integrator neuronsNeural Networks, 2007
- The Introduction of Time-Scales in Reservoir Computing, Applied to Isolated Digits Recognition2007
- Recursive self-organizing network modelsNeural Networks, 2004
- Learning long-term dependencies in NARX recurrent neural networksIEEE Transactions on Neural Networks, 1996
- Learning long-term dependencies with gradient descent is difficultIEEE Transactions on Neural Networks, 1994