Solving the inverse Ising problem by mean-field methods in a clustered phase space with many states

In this work we explain how to properly use mean-field methods to solve the inverse Ising problem when the phase space is clustered, that is, many states are present. The clustering of the phase space can occur for many reasons, e.g., when a system undergoes a phase transition, but also when data are collected in different regimes (e.g., quiescent and spiking regimes in neural networks). Mean-field methods for the inverse Ising problem are typically used without taking into account the eventual clustered structure of the input configurations and may lead to very poor inference (e.g., in the low-temperature phase of the Curie-Weiss model). In this work we explain how to modify mean-field approaches when the phase space is clustered and we illustrate the effectiveness of our method on different clustered structures (low-temperature phases of Curie-Weiss and Hopfield models).

Figure 1

Inference of couplings in the CW model with $N=100$ and two different values of the number $M$ of input configurations. We see that the nMF method with all the input data is good only for $\beta <{\beta }_c=1$. For $\beta >{\beta }_c$ the phase space separates into two states and the nMF method with two clusters gives much better performance (although it fails badly at high temperature). Inference methods, like PLM and nMF with density clustering, that take correctly into account the clustering of input configurations provide the best estimate in the entire temperature range, both above and below the transition temperature. In the left inset, we show how the inferred magnetic field at $\beta =1.6$ decreases when increasing the number of samples ($M\in [{10}^3,{10}^6]$) used for the inference process via nMF with $K$-means clustering and $K=2$. In the right inset the same inferred magnetic field is plotted vs $\beta$, for $M={10}^4,{10}^5$.

Figure 2

Main panels: errors in inferring couplings in Hopfield models with $P=2$ uncorrelated (top), correlated (center), and anticorrelated (bottom) patterns. The comparison is between MF methods with clustered data (either $K$-means or density clustering) and PLM. In the inset of the top panel, we show that the likelihood of the clustering algorithm suggests to take one cluster below ${\beta }_c\approx 1.1$ and four clusters above ${\beta }_c$. In the inset of the bottom panel we show the magnetic field inferred by nMF + clustering, which is very small in both phases.

Figure 3

Errors on inferring couplings in the Hopfield model with three patterns (and thus six minima). We observe again that our algorithm, based on MF methods applied to clustered data, achieves its best performance when input data are split in six clusters. We also put for comparison the results obtained when the clustering is done many times with different initial conditions (labeled “many IC”) and then we picked the clustering having the largest likelihood. In this case, the error matches the error obtained when putting each configuration in the correct cluster. We can see that our method performs its best at almost any $\beta$ value, but at few points where it is particularly difficult to find the best clustering. In the inset we see that likelihood maximization suggests to use one cluster for $\beta <{\beta }_c$ and six clusters for $\beta >{\beta }_c$.