Locality of Global Stochastic Interaction in Directed Acyclic Networks
Author:
- 1 December 2002
- journal article
- Published by MIT Press in Neural Computation
- Vol. 14 (12) , 2959-2980
Abstract
The hypothesis of invariant maximization of interaction (IMI) is formulated within the setting of random fields. According to this hypothesis, learning processes maximize the stochastic interaction of the neurons subject to constraints. We consider the extrinsic constraint in terms of a fixed input distribution on the periphery of the network. Our main intrinsic constraint is given by a directed acyclic network structure. First mathematical results about the strong relation of the local information flow and the global interaction are stated in order to investigate the possibility of controlling IMI optimization in a completely local way. Furthermore, we discuss some relations of this approach to the optimization according to Linsker's Infomax principle.Keywords
This publication has 16 references indexed in Scilit:
- An Information-Geometric Approach to a Theory of Pragmatic StructuringThe Annals of Probability, 2002
- Information geometry on hierarchy of probability distributionsIEEE Transactions on Information Theory, 2001
- Redundancy reduction revisitedNetwork: Computation in Neural Systems, 2001
- Natural Gradient Works Efficiently in LearningNeural Computation, 1998
- An Information-Maximization Approach to Blind Separation and Blind DeconvolutionNeural Computation, 1995
- Unsupervised LearningNeural Computation, 1989
- Receptive fields and functional architecture of monkey striate cortexThe Journal of Physiology, 1968
- Receptive fields, binocular interaction and functional architecture in the cat's visual cortexThe Journal of Physiology, 1962
- Some informational aspects of visual perception.Psychological Review, 1954
- On Information and SufficiencyThe Annals of Mathematical Statistics, 1951