Information geometry for multiparameter models: new perspectives on the origin of simplicity
Top Cited Papers
- 28 December 2022
- journal article
- review article
- Published by IOP Publishing in Reports on Progress in Physics
- Vol. 86 (3) , 035901
Abstract
Complex models in physics, biology, economics, and engineering are often sloppy, meaning that the model parameters are not well determined by the model predictions for collective behavior. Many parameter combinations can vary over decades without significant changes in the predictions. This review uses information geometry to explore sloppiness and its deep relation to emergent theories. We introduce the model manifold of predictions, whose coordinates are the model parameters. Its hyperribbon structure explains why only a few parameter combinations matter for the behavior. We review recent rigorous results that connect the hierarchy of hyperribbon widths to approximation theory, and to the smoothness of model predictions under changes of the control variables. We discuss recent geodesic methods to find simpler models on nearby boundaries of the model manifold—emergent theories with fewer parameters that explain the behavior equally well. We discuss a Bayesian prior which optimizes the mutual information between model parameters and experimental data, naturally favoring points on the emergent boundary theories and thus simpler models. We introduce a ‘projected maximum likelihood’ prior that efficiently approximates this optimal prior, and contrast both to the poor behavior of the traditional Jeffreys prior. We discuss the way the renormalization group coarse-graining in statistical mechanics introduces a flow of the model manifold, and connect stiff and sloppy directions along the model manifold with relevant and irrelevant eigendirections of the renormalization group. Finally, we discuss recently developed ‘intensive’ embedding methods, allowing one to visualize the predictions of arbitrary probabilistic models as low-dimensional projections of an isometric embedding, and illustrate our method by generating the model manifold of the Ising model.Keywords
All Related Versions
Funding Information
- Simons Foundation
- Aspen Center for Physics (NSF PHY-1607611)
- National Institutes of Health (NIH R35 GM138341)
- National Science Foundation (NSF CMMT-1753357)
This publication has 94 references indexed in Scilit:
- Correlations in ion channel expression emerge from homeostatic tuning rulesProceedings of the National Academy of Sciences, 2013
- A Bayesian approach to targeted experiment designBioinformatics, 2012
- Sloppy models, parameter uncertainty, and the role of experimental designMolecular BioSystems, 2010
- A Wnt Oscillator Model for SomitogenesisBiophysical Journal, 2010
- Mutational robustness can facilitate adaptationNature, 2010
- Evidence that Fold-Change, and Not Absolute Level, of β-Catenin Dictates Wnt SignalingMolecular Cell, 2009
- Sloppiness, robustness, and evolvability in systems biologyCurrent Opinion in Biotechnology, 2008
- Universally Sloppy Parameter Sensitivities in Systems Biology ModelsPLOS Computational Biology, 2007
- Estimating the Dimension of a ModelThe Annals of Statistics, 1978
- Transformation of Gorkov's equation for type II superconductors into transport-like equationsThe European Physical Journal A, 1968