Predicting nucleation using machine learning in the Ising model

We use a convolutional neural network (CNN) and two logistic regression models to predict the probability of nucleation in the two-dimensional Ising model. The three methods successfully predict the probability for the nearest-neighbor Ising model for which classical nucleation is observed. The CNN outperforms the logistic regression models near the spinodal of the long-range Ising model, but the accuracy of its predictions decreases as the quenches approach the spinodal. An occlusion analysis suggests that this decrease is due to the vanishing difference between the density of the nucleating droplet and the background. Our results are consistent with the general conclusion that predictability decreases near a critical point.

Figure 1

The time dependence of the nucleation probability $p_{\mathrm{nuc}}$. The time $t=0$ corresponds to the time at which $m$ first satisfies the condition $m\le m_0$. Nucleation occurred before $t=0$. Results are shown for $R=1$, $\mathrm{\Delta }h=0.70$, $L=10$ ($•$) and $R=10$, $\mathrm{\Delta }h=0.055$, $L=100$ ($\circ$).

Figure 2

The nucleation probability $p_{\mathrm{nuc}}$ determined by the intervention method versus the magnetization $m$ for (a) $R=1$, $h=0.50$, $L=100$ and (b) for $R=10$, $h=1.21$, $L=100$. Recall that $m>0$ in the metastable state. For classical nucleation, $m$ has reasonable correlation with the probability of nucleation. For $R=10$ there is no obvious correlation between $m$ and $p_{\mathrm{nuc}}$.

Figure 3

Dependence of the mean-square error of spin-based logistic regression ($▾$), features-based logistic regression ($•$), and CNN ($\blacksquare$) on the linear dimension $L$ of the nearest-neighbor Ising model. The performance of the two logistic regression methods depends only weakly on $L$, but the performance of the CNN decreases with increasing $L$.

Figure 4

Comparison of $p_{\mathrm{nuc}}$ and the values of ${\widehat{p}}_{\mathrm{nuc}}$ generated by the three machine learning models near the spinodal with $\mathrm{\Delta }h=0.06$, $R=10$, and $L=100$. The diagonal line represents perfect prediction. The predictions of features-based logistic regression and CNN are reasonably accurate. The mean-square error of CNN is equal to 0.0046 compared to the mean-square error of 0.0084 for features-based logistic regression.

Figure 5

The performance of the CNN as the magnetic field $h$ approaches $h_s$ for fixed linear dimension $L=200$. The interaction range $R$ is chosen so that the Ginzburg parameter $G$ is constant with $G=6$. As $h$ approaches $h_s$, the performance of the CNN model becomes less robust.

Figure 6

The evolution of the energy per spin $E/N$($•$), the magnetization $m$ ($\circ$), and the number of spins in the largest cluster $s_{max}$ ($\square$) in the Ising model with $R=10$, $L=200$, $\mathrm{\Delta }h=0.06$. The evolution is averaged over 1000 nucleation events.

Figure 7

Comparison of using $m$ ($\circ$) $E/N$ ($\square$), or $s_{max}$ ($•$) as input for determining ${\widehat{p}}_{\mathrm{nuc}}$. The predicted results from the CNN using $s_{max}$ shows some positive correlation with the true value ($\text{MSE}=3.96\times {10}^{-2}$). The evolution of $m$ and $E/N$ does not lead to accurate predictions ($\text{MSE}=9.05\times {10}^{-2}$ and $9.40\times {10}^{-2}$, respectively).

Figure 8

Comparison of the performance of pure spatial ($\blacksquare$, $\text{MSE}=4.43\times {10}^{-3}$), pure temporal ($•$, $\text{MSE}=3.96\times {10}^{-2}$), and spatial $+$ temporal ($\circ$, $\text{MSE}=6.12\times {10}^{-3}$) CNN models. Combining spatial and temporal information made the predictions somewhat less accurate than using spatial information only.

Figure 9

The weights corresponding to each geometric feature in features-based logistic regression (averaged using cross-validation). The most important features are the magnetization $m$ and the first moment $R^{\left(1\right)}$.

Figure 10

Occlusion map for $R=10$, $\mathrm{\Delta }h=0.06$, and $L=200$. The red circle is the average radius of gyration of the biggest cluster, which overlaps with the high-sensitivity region of the occlusion map.

Figure 11

Occlusion sensitivity maps for the Ising model with $R=10$ and $L=200$ for decreasing values of $\mathrm{\Delta }h$, keeping $G$ fixed at $G=6$. The radius of the circular region enclosed by the red line is equal to $R_g$, the mean radius of gyration of the largest cluster. Note that the size of the occlusion sensitive region becomes larger but less intense, similar to the increasing size and decreasing density of the largest cluster.

Figure 12

(a) Dependence of the occlusion sensitivity (red lines) and the density of the nucleating cluster (blue dashed lines) on $r$, the distance from the center of mass of the largest cluster. (b) The radius of gyration of the occlusion sensitivity region is strongly correlated with the radius of gyration of the largest cluster.

Figure 13

Occlusion sensitivity map for the history of the size of the largest cluster as input. The sensitive region extends further into the past as $h\to h_s$.