Recurrent Neural Networks Are Universal Approximators With Stochastic Inputs
- 16 February 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 34 (10) , 7992-8006
Abstract
In this article, we investigate the approximation ability of recurrent neural networks (RNNs) with stochastic inputs in state space model form. More explicitly, we prove that open dynamical systems with stochastic inputs can be well-approximated by a special class of RNNs under some natural assumptions, and the asymptotic approximation error has also been delicately analyzed as time goes to infinity. In addition, as an important application of this result, we construct an RNN-based filter and prove that it can well-approximate finite dimensional filters which include Kalman filter (KF) and Beneš filter as special cases. The efficiency of RNN-based filter has also been verified by two numerical experiments compared with optimal KF.Keywords
Funding Information
- National Natural Science Foundation of China (11961141005)
- Tsinghua University start-up fund
- Tsinghua University Education Foundation Fund (042202008)
This publication has 29 references indexed in Scilit:
- Asynchronous Filtering for Delayed Markovian Jump Systems via Homogeneous Polynomial ApproachIEEE Transactions on Automatic Control, 2019
- Nonlinear Bayesian filtering and learning: a neuronal dynamics for perceptionScientific Reports, 2017
- Hermite Spectral Method to 1-D Forward Kolmogorov Equation and Its Application to Nonlinear Filtering ProblemsIEEE Transactions on Automatic Control, 2013
- Classification of finite-dimensional estimation algebras of maximal rank with arbitrary state–space dimension and mitter conjectureInternational Journal of Control, 2005
- Unscented Filtering and Nonlinear EstimationProceedings of the IEEE, 2004
- Complete classification of finite-dimensional estimation algebras of maximal rankInternational Journal of Control, 2003
- An algorithmic approach to adaptive state filtering using recurrent neural networksIEEE Transactions on Neural Networks, 2001
- Novel approach to nonlinear/non-Gaussian Bayesian state estimationIEE Proceedings F Radar and Signal Processing, 1993
- Approximation by superpositions of a sigmoidal functionMathematics of Control, Signals, and Systems, 1989
- New exact nonlinear filters with large Lie algebrasSystems & Control Letters, 1985