A statistical physics approach to learning curves for the inverse Ising problem
- 1 June 2017
- journal article
- Published by IOP Publishing in Journal of Statistical Mechanics: Theory and Experiment
- Vol. 2017 (6) , 063406
Abstract
Using methods of statistical physics, we analyse the error of learning couplings in large Ising models from independent data (the inverse Ising problem). We concentrate on learning based on local cost functions, such as the pseudo-likelihood method for which the couplings are inferred independently for each spin. Assuming that the data are generated from a true Ising model, we compute the reconstruction error of the couplings using a combination of the replica method with the cavity approach for densely connected systems. We show that an explicit estimator based on a quadratic cost function achieves minimal reconstruction error, but requires the length of the true coupling vector as prior knowledge. A simple mean field estimator of the couplings which does not need such knowledge is asymptotically optimal, i.e. when the number of observations is much larger than the number of spins. Comparison of the theory with numerical simulations shows excellent agreement for data generated from two models with random couplings in the high temperature region: a model with independent couplings (Sherrington–Kirkpatrick model), and a model where the matrix of couplings has a Wishart distribution.Keywords
All Related Versions
This publication has 34 references indexed in Scilit:
- Inverse Ising Inference Using All the DataPhysical Review Letters, 2012
- Ising model for neural data: Model quality and approximate methods for extracting functional connectivityPhysical Review E, 2009
- Identification of direct residue contacts in protein–protein interaction by message passingProceedings of the National Academy of Sciences of the United States of America, 2009
- Small-correlation expansions for the inverse Ising problemJournal of Physics A: Mathematical and Theoretical, 2009
- Statistical physics of pairwise probability modelsFrontiers in Computational Neuroscience, 2009
- Using the principle of entropy maximization to infer genetic interaction networks from gene expression patternsProceedings of the National Academy of Sciences, 2006
- Weak pairwise correlations imply strongly correlated network states in a neural populationNature, 2006
- Mean-field theory of Boltzmann machine learningPhysical Review E, 1998
- Efficient Learning in Boltzmann Machines Using Linear Response TheoryNeural Computation, 1998
- Spatial Interaction and the Statistical Analysis of Lattice SystemsJournal of the Royal Statistical Society Series B: Statistical Methodology, 1974