Information theory: A foundation for complexity science
Top Cited Papers
Open Access
- 27 July 2022
- journal article
- research article
- Published by National Academy of Sciences in Proceedings of the National Academy of Sciences
- Vol. 119 (33)
Abstract
Modeling and inference are central to most areas of science and especially to evolving and complex systems. Critically, the information we have is often uncertain and insufficient, resulting in an underdetermined inference problem; multiple inferences, models, and theories are consistent with available information. Information theory (in particular, the maximum information entropy formalism) provides a way to deal with such complexity. It has been applied to numerous problems, within and across many disciplines, over the last few decades. In this perspective, we review the historical development of this procedure, provide an overview of the many applications of maximum entropy and its extensions to complex systems, and discuss in more detail some recent advances in constructing comprehensive theory based on this inference procedure. We also discuss efforts at the frontier of information-theoretic inference: application to complex dynamic systems with time-varying constraints, such as highly disturbed ecosystems or rapidly changing economies.Keywords
This publication has 65 references indexed in Scilit:
- High-order social interactions in groups of miceeLife, 2013
- Statistical mechanics for natural flocks of birdsProceedings of the National Academy of Sciences, 2012
- Biology, Methodology or Chance? The Degree Distributions of Bipartite Ecological NetworksPLOS ONE, 2011
- Information-theoretic analysis of phenotype changes in early stages of carcinogenesisProceedings of the National Academy of Sciences, 2010
- Maximum entropy models for antibody diversityProceedings of the National Academy of Sciences, 2010
- Pairwise Maximum Entropy Models for Studying Large Biological Systems: When They Can Work and When They Can'tPLOS Computational Biology, 2009
- Identification of direct residue contacts in protein–protein interaction by message passingProceedings of the National Academy of Sciences, 2009
- Using the principle of entropy maximization to infer genetic interaction networks from gene expression patternsProceedings of the National Academy of Sciences, 2006
- Weak pairwise correlations imply strongly correlated network states in a neural populationNature, 2006
- Information Theory and Statistical MechanicsPhysical Review B, 1957