The Stochastic complexity of spin models: how simple are simple spin models?

Authors:
  • ABAlberto Beretta
    Alberto Beretta
  • CBClaudia Battistin
    Claudia Battistin
  • CMClélia de Mulatier
    Clélia de Mulatier
  • IMIacopo Mastromatteo
    Iacopo Mastromatteo
  • MMMatteo Marsili
    Matteo Marsili

Abstract
Models can be simple for different reasons. In science, models are simple when they depend on few variables, they are meant to yield predictions that can be falsified. Parameters are often mutually constrained within scales. They spot symmetries and conservation laws specific of the phenomenon under study. Models in statistical learning are simple because they are easy to evaluate, train and/or to infer. They are simple to interpret in terms of low order dependencies (e.g. graphical models). Their aim is not to uncover fundamental laws but to "generalise well", i.e. to describe well yet unseen data. We show that, for spin models with interactions of arbitrary order, the information theoretic notion of complexity (or simplicity) provided by Minimum Description Length (MDL) conforms with the scientific notion of simplicity rather than with that in machine learning. Simple models in MDL have statistical dependencies concentrated in groups of few variables, they afford predictions on independencies that are easy to falsify. On the contrary, pairwise interacting models, which are often used in statistical learning, appear to be rather complex.

This publication has 13 references indexed in Scilit: