An Information-Geometric Approach to a Theory of Pragmatic Structuring
Author:
- Nihat AyNANihat Ay
Open Access
- 1 January 2002
- journal article
- Published by Institute of Mathematical Statistics in The Annals of Probability
- Vol. 30 (1) , 416-436
Abstract
Within the framework of information geometry, the interaction among units of a stochastic system is quantified in terms of the Kullback–Leibler divergence of the underlying joint probability distribution from an appropriate exponential family. In the present paper, the main example for such a family is given by the set of all factorizable random fields. Motivated by this example, the locally farthest points from an arbitrary exponential family $\mathcal{E}$ are studied. In the corresponding dynamical setting, such points can be generated by the structuring process with respect to $\mathcal{E}$ as a repelling set. The main results concern the low complexity of such distributions which can be controlled by the dimension of $\mathcal{E}$.
Keywords
This publication has 21 references indexed in Scilit:
- Information geometry on hierarchy of probability distributionsIEEE Transactions on Information Theory, 2001
- Information geometry1997
- Detecting higher-order interactions among the spiking events in a group of neuronsBiological Cybernetics, 1995
- Gradient systems in view of information geometryPhysica D: Nonlinear Phenomena, 1995
- Completely integrable gradient systems on the manifolds of Gaussian and multinomial distributionsJapan Journal of Industrial and Applied Mathematics, 1993
- $I$-Divergence Geometry of Probability Distributions and Minimization ProblemsThe Annals of Probability, 1975
- On the Uniform Convergence of Relative Frequencies of Events to Their ProbabilitiesTheory of Probability & Its Applications, 1971
- Information Theory and Statistical MechanicsPhysical Review B, 1957
- On Information and SufficiencyThe Annals of Mathematical Statistics, 1951
- A Mathematical Theory of CommunicationBell System Technical Journal, 1948