Dmitry Panchenko

We consider generic optimal Bayesian inference, namely, models of signal reconstruction where the posterior distribution and all hyperparameters are known. Under a standard assumption on the concentration of the free energy, we show how replica symmetry in the strong sense of concentration of all multioverlaps can be established as a consequence of the Franz-de Sanctis identities; the identities themselves in the current setting are obtained via a novel perturbation coming from exponentially distributed "side-observations" of the signal. Concentration of multioverlaps means that asymptotically the posterior distribution has a particularly simple structure encoded by a random probability measure (or, in the case of binary signal, a non-random probability measure). We believe that such strong control of the model should be key in the study of inference problems with underlying sparse graphical structure (error correcting codes, block models, etc) and, in particular, in the rigorous derivation of replica symmetric formulas for the free energy and mutual information in this context.

We consider the mixed p‐spin mean‐field spin glass model with Ising spins and investigate its free energy in the spirit of the TAP approach, named after Thouless, Anderson, and Palmer [67]. More precisely, we define and compute the generalized TAP correction, and establish the corresponding generalized TAP representation for the free energy. In connection with physicists’ replica theory, we introduce the notion of generalized TAP states, which are the maximizers of the generalized TAP free energy, and show that their order parameters match the order parameter of the ancestor states in the Parisi ansatz. We compute the critical point equations of the TAP free energy that generalize the classical TAP equations for pure states. Furthermore, we give an exact description of the region where the generalized TAP correction is replica symmetric, in which case it coincides with the classical TAP correction, and show that Plefka's condition is necessary for this to happen. In particular, our result shows that the generalized TAP correction is not always replica symmetric on the points corresponding to the Edwards‐Anderson parameter. © 2022 Wiley Periodicals LLC.

In this paper we analyze, for a model of linear regression with gaussian covariates, the performance of a Bayesian estimator given by the mean of a log-concave posterior distribution with gaussian prior, in the high-dimensional limit where the number of samples and the covariates' dimension are large and proportional. Although the high-dimensional analysis of Bayesian estimators has been previously studied for Bayesian-optimal linear regression where the correct posterior is used for inference, much less is known when there is a mismatch. Here we consider a model in which the responses are corrupted by gaussian noise and are known to be generated as linear combinations of the covariates, but the distributions of the ground-truth regression coefficients and of the noise are unknown. This regression task can be rephrased as a statistical mechanics model known as the Gardner spin glass, an analogy which we exploit. Using a leave-one-out approach we characterize the mean-square error for the regression coefficients. We also derive the log-normalizing constant of the posterior. Similar models have been studied by Shcherbina and Tirozzi and by Talagrand, but our arguments are much more straightforward. An interesting consequence of our analysis is that in the quadratic loss case, the performance of the Bayesian estimator is independent of a global "temperature" hyperparameter and matches the ridge estimator: sampling and optimizing are equally good.

We consider the mixed $p$-spin mean-field spin glass model with Ising spins and investigate its free energy in the spirit of the TAP approach, named after Thouless, Anderson, and Palmer. More precisely, we define and compute the generalized TAP correction, and establish the corresponding generalized TAP representation for the free energy. In connection with physicists' replica theory, we introduce the notion of generalized TAP states, which are the maximizers of the generalized TAP free energy, and show that their order parameters match the order parameter of the ancestor states in the Parisi ansatz. We compute the critical point equations of the TAP free energy that generalize the classical TAP equations for pure states. Furthermore, we give an exact description of the region where the generalized TAP correction is replica symmetric, in which case it coincides with the classical TAP correction, and show that Plefka's condition is necessary for this to happen. In particular, our result shows that the generalized TAP correction is not always replica symmetric on the points corresponding to the Edwards-Anderson parameter.

We study the free energy of mixed $p$-spin spin glass models enriched with an additional magnetic field given by the canonical Gaussian field associated with a Ruelle probability cascade. We prove that this free energy converges to the Hopf-Lax solution of a certain Hamilton-Jacobi equation. Using this result, we give a new representation of the free energy of mixed $p$-spin models with soft spins.

In a recent paper [14], we developed the generalized TAP approach for mixed $p$-spin models with Ising spins at positive temperature. Here we extend these results in two directions. We find a simplified representation for the energy of the generalized TAP states in terms of the Parisi measure of the model and, in particular, show that the energy of all states at a given distance from the origin is the same. Furthermore, we prove the analogues of the positive temperature results at zero temperature, which concern the ground-state energy and the organization of ground-state configurations in space.

In [Physical Magazine, 35(3):593-601, 1977], Thouless, Anderson, and Palmer derived a representation for the free energy of the Sherrington-Kirkpatrick model, called the TAP free energy, written as the difference of the energy and entropy on the extended configuration space of local magnetizations with an Onsager correction term. In the setting of mixed $p$-spin models with Ising spins, we prove that the free energy can indeed be written as the supremum of the TAP free energy over the space of local magnetizations whose Edwards-Anderson order parameter (self-overlap) is to the right of the support of the Parisi measure. Furthermore, for generic mixed $p$-spin models, we prove that the free energy is equal to the TAP free energy evaluated on the local magnetization of any pure state.