On the Relative Expressiveness of Bayesian and Neural Networks
Preprint
Abstract
A neural network computes a function. A central property of neural networks is that they are "universal approximators:" for a given continuous function, there exists a neural network that can approximate it arbitrarily well, given enough neurons (and some additional assumptions). In contrast, a Bayesian network is a model, but each of its queries can be viewed as computing a function. In this paper, we identify some key distinctions between the functions computed by neural networks and those by marginal Bayesian network queries, showing that the former are more expressive than the latter. Moreover, we propose a simple augmentation to Bayesian networks (a testing operator), which enables their marginal queries to become "universal approximators."Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- Human-level intelligence or animal-like abilities?Communications of the ACM, 2018
- A Fast Learning Algorithm for Deep Belief NetsNeural Computation, 2006
- Noisy-or classifierInternational Journal of Intelligent Systems, 2006
- On the revision of probabilistic beliefs using uncertain evidenceArtificial Intelligence, 2005
- A differential approach to inference in Bayesian networksJournal of the ACM, 2003
- Multilayer feedforward networks with a nonpolynomial activation function can approximate any functionNeural Networks, 1993
- An analysis of first-order logics of probabilityArtificial Intelligence, 1990
- Approximation by superpositions of a sigmoidal functionMathematics of Control, Signals, and Systems, 1989
- Multilayer feedforward networks are universal approximatorsNeural Networks, 1989
- The perceptron: A probabilistic model for information storage and organization in the brain.Psychological Review, 1958