Machine learning vortices at the Kosterlitz-Thouless transition

Efficient and automated classification of phases from minimally processed data is one goal of machine learning in condensed-matter and statistical physics. Supervised algorithms trained on raw samples of microstates can successfully detect conventional phase transitions via learning a bulk feature such as an order parameter. In this paper, we investigate whether neural networks can learn to classify phases based on topological defects. We address this question on the two-dimensional classical XY model which exhibits a Kosterlitz-Thouless transition. We find significant feature engineering of the raw spin states is required to convincingly claim that features of the vortex configurations are responsible for learning the transition temperature. We further show a single-layer network does not correctly classify the phases of the XY model, while a convolutional network easily performs classification by learning the global magnetization. Finally, we design a deep network capable of learning vortices without feature engineering. We demonstrate the detection of vortices does not necessarily result in the best classification accuracy, especially for lattices of less than approximately 1000 spins. For larger systems, it remains a difficult task to learn vortices.

Figure 1

Example of a vortex and antivortex in the XY model on the lattice. A vortex has winding number $k=1$, while an antivortex has $k=-1$.

Figure 2

Estimators of the XY model on a $L\times L$ lattice with periodic boundary conditions computed via Monte Carlo sampling. (a) The helicity modulus for various lattice sizes $L$. The estimated critical point ${\tilde{T}}_{\mathrm{KT}}$ is determined by the Nelson-Kosterlitz universal jump where the helicity modulus $\mathrm{{\rm Y}}$ intersects the line $\frac{2T}{\pi }$. The inset shows how ${\tilde{T}}_{\mathrm{KT}}$ scales with ${(logL)}^{-2}$ towards the thermodynamic $T_{\mathrm{KT}}$ shown by the black dashed line. (b) The nonzero magnetization present in the finite-size XY model. The magnetization vanishes as $L^{-\frac{1}{8}}$ in the thermodynamic limit with the scaling shown in the inset.

Figure 3

Finite-size scaling of the predicted $T_{\mathrm{KT}}$ for FFNN and CNN trained on either (a) raw spin configurations or (b) the vorticity. In either case the FFNN performs worse than the CNN according to the test classification accuracy (insets). The critical temperature is determined by the point where the sigmoid output, as a function of temperature, crosses 0.5. Each data point and variance is obtained by training ten networks with stochastic gradient descent until the validation loss function fails to improve after 50 epochs (early stopping).

Figure 4

The learning by confusion scheme for a CNN applied to (a) raw spin configurations and (b) vorticity configurations. The test accuracy is expected to form a $\vee \vee$ shape with the center peak at $T^{*}=T_{\mathrm{KT}}$. (c) The peak in specific heat $C_v$ compared to the peak of the test accuracy for a system of size $L=64$. The dashed vertical line shows the thermodynamic $T_{\mathrm{KT}}$.

Figure 5

Visual representation of how the custom network architecture can compute the vorticity. We denote the convolution operation with $\otimes$ and ignore biases for the purpose of the diagram. Applying the four $2\times 2$ filters $K_i$ partitions the data into four $L\times L$ arrays where each element is an angle difference in one lattice direction $\mathrm{\Delta }{\theta }_{ij}$. The angle differences are then converted into the range $\mathrm{\Delta }{\theta }_{ij}\in [-\pi ,\pi )$ by applying the sawtooth function from Eq. (5). A single $1\times 1$ convolution filter with weights $w=[1,1,1,1]$ and zero biases then sums the four shifted angle differences into the vorticity.

Figure 6

The loss function from Eq. (4) evaluated on the test set for three variations of the custom architecture for various lattice sizes $L$. For small $L<16$, the fixed network with hard-coded weights performs poorly compared to the others. For large $L>32$, the fixed network performs best, possibly due to a reduced number of trainable parameters. The inset shows a magnified region for $32\le L\le 72$.

Figure 7

Histogram of the values of the vortex layer from Fig. 5 which (ideally) computes the vorticity for (a) the network initialized to compute vorticity and (b) the randomly initialized network. In (a), we see for small $L$ that the vorticity is not quantized, indicating that the network did not learn to compute the local vorticity; however, for large $L$, the histogram looks as expected for vortex detection. Conversely, the distribution in (b) appears to be unrelated to vortices for any $L$.