A Hierarchical Bayesian Linear Regression Model with Local Features for Stochastic Dynamics Approximation
Preprint
Abstract
One of the challenges in model-based control of stochastic dynamical systems is that the state transition dynamics are involved, and it is not easy or efficient to make good-quality predictions of the states. Moreover, there are not many representational models for the majority of autonomous systems, as it is not easy to build a compact model that captures the entire dynamical subtleties and uncertainties. In this work, we present a hierarchical Bayesian linear regression model with local features to learn the dynamics of a micro-robotic system as well as two simpler examples, consisting of a stochastic mass-spring damper and a stochastic double inverted pendulum on a cart. The model is hierarchical since we assume non-stationary priors for the model parameters. These non-stationary priors make the model more flexible by imposing priors on the priors of the model. To solve the maximum likelihood (ML) problem for this hierarchical model, we use the variational expectation maximization (EM) algorithm, and enhance the procedure by introducing hidden target variables. The algorithm yields parsimonious model structures, and consistently provides fast and accurate predictions for all our examples involving large training and test sets. This demonstrates the effectiveness of the method in learning stochastic dynamics, which makes it suitable for future use in a paradigm, such as model-based reinforcement learning, to compute optimal control policies in real time.Keywords
All Related Versions
This publication has 18 references indexed in Scilit:
- Data-Driven Identification of Parametric Partial Differential EquationsSIAM Journal on Applied Dynamical Systems, 2019
- Linear Model Regression on Time-series Data: Non-asymptotic Error Bounds and Applications2018
- Regional models: A new approach for nonlinear system identification via clustering of the self-organizing mapNeurocomputing, 2015
- Real-time model learning using Incremental Sparse Spectrum Gaussian Process RegressionNeural Networks, 2013
- Sparse On-Line Gaussian ProcessesNeural Computation, 2002
- Model complexity control for regression using VC generalization boundsIEEE Transactions on Neural Networks, 1999
- Constructive Incremental Learning from Only Local InformationNeural Computation, 1998
- Neural networks for control systems—A surveyAutomatica, 1992
- Identification and control of dynamical systems using neural networksIEEE Transactions on Neural Networks, 1990
- Maximum Likelihood from Incomplete Data Via the EM AlgorithmJournal of the Royal Statistical Society Series B: Statistical Methodology, 1977