Inverse Spin Glass and Related Maximum Entropy Problems

If we have a system of binary variables and we measure the pairwise correlations among these variables, then the least structured or maximum entropy model for their joint distribution is an Ising model with pairwise interactions among the spins. Here we consider inhomogeneous systems in which we constrain, for example, not the full matrix of correlations, but only the distribution from which these correlations are drawn. In this sense, what we have constructed is an inverse spin glass: rather than choosing coupling constants at random from a distribution and calculating correlations, we choose the correlations from a distribution and infer the coupling constants. We argue that such models generate a block structure in the space of couplings, which provides an explicit solution of the inverse problem. This allows us to generate a phase diagram in the space of (measurable) moments of the distribution of correlations. We expect that these ideas will be most useful in building models for systems that are nonequilibrium statistical mechanics problems, such as networks of real neurons.

Figure 1

Maximum entropy for independent spins. The distribution of local magnetization is shown in the left inset. In the main panel we show the field distributions ${\mathfrak{p}}_{\mathsf{2}}(h)$ (in black) and ${\mathfrak{p}}_{\text{all}}(h)$ (in red) from the maximum entropy solution with two moments and all moments of the local magnetization, respectively. The distribution ${\mathfrak{p}}_{\mathsf{2}}(h)$ is given by two delta peaks at $h=h_{\mathit{A}}$ and $h=h_{\mathit{B}}$. In the right inset we show the entropy per spin $s_{\mathsf{2}}$ (in black) in the two moment case vs the fraction $x$ of spins in group $A$, compared with the entropy in the all moment case $s_{\text{all}}$ (in red). The optimal value $x_{*}$ is also marked.

Figure 2

Contour plot of the smallest eigenvalue $\mathrm{\Lambda }$ of the free energy Hessian as a function of $\ln C_1,\ln C_2$ for the correlated spin case in the allowed region $C_2\ge C_1^2$. There is a high temperature phase (in red), where $C_1=\mathcal{O}(1/N)$, $C_2=\mathcal{O}(1/N^2)$, a low-temperature phase (in blue) where $C_1=\mathcal{O}(1)$, $C_2=\mathcal{O}(1)$, and a critical regime where $C_1=\mathcal{O}(1/\sqrt{N})$, $C_2=\mathcal{O}(1/N)$. Scale bars on the $x$ and $y$ axis represent one unit of $\ln C_1$ and $\ln C_2$, respectively. In the high temperature phase and in the critical regime, $\mathrm{\Lambda }$ is a function of the scaled correlations $N{C}_1$, $N^2{C}_2$ and $\sqrt{N}{C}_1$, $N{C}_2$, respectively.