A model-learner pattern for bayesian reasoning
Authors:
Open Access
- 23 January 2013
- journal article
- Published by Association for Computing Machinery (ACM) in ACM SIGPLAN Notices
- Vol. 48 (1) , 403-416
Abstract
A Bayesian model is based on a pair of probability distributions, known as the prior and sampling distributions. A wide range of fundamental machine learning tasks, including regression, classification, clustering, and many others, can all be seen as Bayesian models. We propose a new probabilistic programming abstraction, a typed Bayesian model, which is based on a pair of probabilistic expressions for the prior and sampling distributions. A sampler for a model is an algorithm to compute synthetic data from its sampling distribution, while a learner for a model is an algorithm for probabilistic inference on the model. Models, samplers, and learners form a generic programming pattern for model-based inference. They support the uniform expression of common tasks including model testing, and generic compositions such as mixture models, evidence-based model averaging, and mixtures of experts. A formal semantics supports reasoning about model equivalence and implementation correctness. By developing a series of examples and three learner implementations based on exact inference, factor graphs, and Markov chain Monte Carlo, we demonstrate the broad applicability of this new programming pattern.Keywords
This publication has 33 references indexed in Scilit:
- Quantitative Analysis With the Probabilistic Model Checker PRISMElectronic Notes in Theoretical Computer Science, 2006
- FUNCTIONAL PEARLS: Probabilistic functional programming in HaskellJournal of Functional Programming, 2005
- Abstraction and refinement in probabilistic systemsACM SIGMETRICS Performance Evaluation Review, 2005
- A probabilistic language based upon sampling functionsACM SIGPLAN Notices, 2005
- Stochastic lambda calculus and monads of probability distributionsACM SIGPLAN Notices, 2002
- Bayesian model averaging: a tutorial (with comments by M. Clyde, David Draper and E. I. George, and a rejoinder by the authorsStatistical Science, 1999
- Hierarchical Mixtures of Experts and the EM AlgorithmNeural Computation, 1994
- Adaptive Mixtures of Local ExpertsNeural Computation, 1991
- Semantics of probabilistic programsJournal of Computer and System Sciences, 1981
- Monte Carlo sampling methods using Markov chains and their applicationsBiometrika, 1970