Discovering phases, phase transitions, and crossovers through unsupervised machine learning: A critical examination

We apply unsupervised machine learning techniques, mainly principal component analysis (PCA), to compare and contrast the phase behavior and phase transitions in several classical spin models—the square- and triangular-lattice Ising models, the Blume-Capel model, a highly degenerate biquadratic-exchange spin-1 Ising (BSI) model, and the two-dimensional $XY$ model—and we examine critically what machine learning is teaching us. We find that quantified principal components from PCA not only allow the exploration of different phases and symmetry-breaking, but they can distinguish phase-transition types and locate critical points. We show that the corresponding weight vectors have a clear physical interpretation, which is particularly interesting in the frustrated models such as the triangular antiferromagnet, where they can point to incipient orders. Unlike the other well-studied models, the properties of the BSI model are less well known. Using both PCA and conventional Monte Carlo analysis, we demonstrate that the BSI model shows an absence of phase transition and macroscopic ground-state degeneracy. The failure to capture the “charge” correlations (vorticity) in the BSI model ($XY$ model) from raw spin configurations points to some of the limitations of PCA. Finally, we employ a nonlinear unsupervised machine learning procedure, the “autoencoder method,” and we demonstrate that it too can be trained to capture phase transitions and critical points.

This article appears in the following collection:

Figure 1

Top row: Spin configuration snapshots from conventional Metropolis Monte Carlo simulations of the BSI model with fixed $K=1, J=0$ and linear lattice size $L=40$. At high temperatures there is a random mix of the three spin possibilities $S_i=0,\pm 1$. As $T$ decreases below $K$, sites occupied by $S_i=\pm 1$ are gradually surrounded by near-neighbor vacancy sites $S_i=0$. Bottom row: Similar spin configuration snapshots with $J=0.1$. The evolution from $T=2.0$ (f) to $T=0.3$ (e) is similar to that at $J=0.0$ [(c)–(b)]. However, in this case an ordered phase emerges (d) at the lowest $T=0.1$. In all BSI calculations reported in this paper, $K$ is fixed to 1 as the energy scale.

Figure 2

Schematic structure of an autoencoder with two hidden neurons. CNNs are treated as the encoder and the decoder in this work.

Figure 3

PCA results for the 2D square-lattice Ising model. (a) Relative variances ${\tilde{\lambda }}_n$ obtained from the raw Ising configurations, with the horizontal axis indicating corresponding component labels. (b) Projection of the raw Ising configurations onto the plane of the two leading principal components. The color bar on the right indicates the temperature in units of $J$. (c) The normalized quantified first leading component as a function of temperature. (d) The quantified second leading component as a function of temperature.

Figure 4

(a) Visualization of the weight vector corresponding to the first leading component. (b) Visualization of the weight vector corresponding to the second leading component. (c) The vector ${\mathbf{w}}_2^{\prime }$ defined by Eq. (10). In (a)–(c) the lattice size $L=50$. (d) Peaks ($T^{*}$) returned from the quantified second leading component as a function of the inverse of the lattice dimension ($1/L$). The extrapolation was performed using a linear least-squares fit.

Figure 5

PCA results for the BCM at fixed $J=1.0,T=1.0$ and sweeping $\mathrm{\Delta }$. (a) Relative variances obtained from the raw Blume-Capel configurations, with the horizontal axis indicating the corresponding component label. (b) Projection of the raw Blume-Capel configurations onto the plane of the leading two principal components. The color bar on the right indicates $\mathrm{\Delta }$ in units of $J$. (c) The normalized quantified first leading component as a function of $\mathrm{\Delta }$. (d) The quantified second leading component as a function $\mathrm{\Delta }$. Peaks provide estimates of the transition point ${\mathrm{\Delta }}_c$.

Figure 6

PCA results for the BCM at fixed $J=1.0,T=0.4$. (a) Relative variances. (b) Projection of the raw Blume-Capel configurations onto the plane of two leading principal components. The color bar on the right indicates $\mathrm{\Delta }$ in units of $J$. (c) The normalized quantified first leading component as a function of $\mathrm{\Delta }$. (d) The quantified second leading component as a function of $\mathrm{\Delta }$. Peaks provide estimates of the transition value ${\mathrm{\Delta }}_c$.

Figure 7

(a) Energy as a function of temperature with (blue) or without (red) the $J$ term. (b) Specific heat as a function of temperature with (green, brown) or without (blue, yellow) the $J$ term. Finite-size effect is verified to be negligible. Sharp peaks agree well with the Binder ratio crossing in Fig. 8. (c) Structure factor as a function of temperature with different system sizes. $J$ is fixed to 0. (d) Entropy as a function of temperature with (blue) or without (red) the $J$ term. Ground-state entropy for the red curve is 0.584.

Figure 8

Monte Carlo results for the BSI model with fixed $K=1.0,J=0.1$. (a) Absolute magnetization as a function of temperature. (b) Susceptibility as a function of temperature. Sharp peaks agree well with the Binder ratio crossing. (c) Binder ratio as a function of temperature with varied system sizes. The critical temperature is $T_c=0.163$.

Figure 9

PCA results for the BSI model at lattice size $L=30$. Color bars of scatter plots indicate the temperature of the samples in units of $K$. Top row: The BSI model with fixed $K=1.0,J=0$. (a) Relative variances obtained from the raw BSI configurations $\left\{{\mathbf{S}}_i\right\}$ (red) and the squared spin configurations $\left\{{\mathbf{S}}_i^2\right\}$ (blue). (b) Projection of the raw BSI configurations $\left\{{\mathbf{S}}_i\right\}$ onto the plane of the leading two principal components. (c) Projection of the squared spin configurations $\left\{{\mathbf{S}}_i^2\right\}$ onto the plane of the leading two principal components. Bottom row: The BSI model with fixed $K=1.0,J=0.1$. (d), (e), and (f) follow the same definitions as (a), (b), and (c).

Figure 10

Four distinct ground states for the BSI model, with fixed $K=1,J=0.1,N=40\times 40$. For clarity, we only display a $4\times 4$ block for each ground state.

Figure 11

Top row: PCA results for the antiferromagnetic (fully frustrated) TLIM, with fixed $J=-1$. (a) Relative variances. $L=30$. (b) Projection onto the plane of the leading two principal components. The color bar indicates the temperature in units of $\left|J\right|$. $L=30$. (c) The normalized quantified first leading component as a function of temperature. (d) The normalized quantified second leading component as a function of temperature. (e) The weight vector corresponding to the first leading component. (f) The weight vector corresponding to the second leading component. See the text for an interpretation.

Figure 12

PCA results for the $XY$ model with fixed $J=1.0$ at $L=30$. Top row: Feed angle configurations $\{cos\left({\theta }_i\right),sin\left({\theta }_i\right)\}$ into PCA. (a) Relative variances. (b) Projection onto the plane of the leading two principal components. The color bar indicates the temperature in units of $J$. Each temperature has 100 scatter points. (c) $〈\sqrt{p_1^2+p_2^2}〉/L$ vs temperature. The dashed line marks the true $T_c$. Bottom row: (d) Relative variances. Red (blue) triangles, with the label $\left\{{\mathbf{V}}_i\right\}$ ($\left\{{\mathbf{V}}_i^2\right\}$), indicate results by feeding the raw vorticity (squared vorticity) configurations into PCA. (e) Projection of the squared vorticity configurations onto the leading two principal components. The color bar indicates temperature, and each temperature has 100 scatter points. (f) $〈|p_1|〉/L$ vs temperature. The dashed line marks the true $T_c$.

Figure 13

Ising model autoencoder results. $J=1.0$ is fixed and the original lattice size is $N=40\times 40$. (a) Spin configuration (above) and reconstructions (below) at the indicated temperatures using 200 hidden neurons. (b) Encoding of the raw Ising configurations $\left\{{\mathbf{S}}_i\right\}$ onto the plane of the two hidden neuron activations ($h_1,h_2$). Color bar indicates the temperature of the sample in units of $J$. Each temperature has 100 scatter points. (c) Encoding of the raw Ising configurations $\left\{{\mathbf{S}}_i\right\}$ using a single hidden neuron activation ($h^{*}$) as a function of temperature. Each data point is averaged over $n=10000$ samples of the same temperature.

Figure 14

Autoencoder architecture. To use 200 hidden neurons, we choose to follow the path from black to red to black, and to use 2 or 1 hidden neuron, we choose to follow the path from black to blue to black. We use raw spin configurations as the input data, with fixed size $N=40\times 40$.