Berezinskii-Kosterlitz-Thouless transition from neural network flows

We adopt the neural network (NN) flow method to study the Berezinskii-Kosterlitz-Thouless (BKT) phase transitions of the two-dimensional $q$-state clock model with $q\ge 4$. The NN flow consists of a sequence of the same units that proceed with the flow. This unit is a variational autoencoder trained by the data of Monte Carlo configurations in unsupervised learning. To gauge the difference among the ensembles of Monte Carlo configurations at different temperatures and the uniqueness of the ensemble of NN-flow states, we adopt the Jensen-Shannon divergence (JSD) as the information-distance measure “thermometer.” This JSD thermometer compares the probability distribution functions of the mean spin value of two ensembles of states. Our results show that the NN flow will flow an arbitrary spin state to some state in a fixed-point ensemble of states. The corresponding JSD of the fixed-point ensemble takes a unique profile with peculiar features, which can help to identify the critical temperatures of BKT phase transitions of the underlying Monte Carlo configurations.

Figure 1

(a) Magnetization and (b) magnetic susceptibility for the $q$-state clock model are obtained from Monte Carlo simulations on a $40\times 40$ lattice.

Figure 2

Schematic structure of the NN flow (left) with the substructure of each unit, i.e., a (variational) AE (right). We consider VAE so that the hidden layer vector $z$ is stochastic with Gaussian distributions.

Figure 3

Naive analog between RG flow and neural network flow. The decimation of the renormalization flow is in analogy to the encoder and its inverse to the decoder. Therefore, the stochastic hidden layer plays the role of the mean field in the renormalization flow. As a result, the different choices of the stochastic distribution for the hidden layer may correspond to the different mean fields and yield different fixed-point states.

Figure 4

Probability density functions (PDF) for the mean spin value $M_n$ of $q=4$ clock model on $20\times 20$ lattice. Blue (shading): the PDFs for the temperature ensembles of Monte Carlo simulated configurations at (a) $T=1$ (below $T_c$), (b) $T=1.24$ (near $T_c$), and (c) $T=1.8$ (above $T_c$). Red (line): the PDFs of the corresponding NN-flow ensemble after 100 iterations. It shows that the NN-flow states approach some states in a fixed-point ensemble of states.

Figure 5

Jensen-Shannon divergence $\mathrm{JSD}\left[P_{T_p}\right|\left|Q_T\right]$ of the (a) $q=4$ (dashed line) and 8 (solid line) and (b) $q=5$ (solid line), 6 (dashed line), and 7 (dotted line) clock models on a $40\times 40$ lattice. Here $T$ runs all the temperature bins in $0<T<4$, and $T_p$ runs for three specific temperatures: $T_{\mathrm{low}}=0.4$ (red color), $T_{\mathrm{mid}}=0.8$ (blue color), and $T_{\mathrm{high}}=1.6$ (green color). The JSDs can serve as a good “thermometer” because the minimum of the JSDs sits right on the corresponding $T_p$.

Figure 6

NN-flow $\mathrm{JSD}\left[P_{T_{\mathrm{flow}}}\right|\left|Q_T\right]$ of $q=2$ clock model on a $20\times 20$ lattice. $P_{T_{\mathrm{flow}}}$ is the PDF of the mean spin value of NN-flow states after one iteration (dashes line) and 100 iterations (solid line) of the initial Monte Carlo ensemble at temperature $T_{\mathrm{flow}}$, and $Q_T$ is the reference PDF based on Monte Carlo ensemble at temperature $T$. For convenience, we choose $T_{\mathrm{flow}}=T_{\mathrm{low}}$ (red), $T_{\mathrm{mid}}$ (blue), and $T_{\mathrm{high}}$ (green) of Fig. 5. We see that there is a unique pattern for NN-flow JSDs with its minimum sitting inside the narrow window of $[{\left(T_c\right)}_{\mathrm{exact}},{\left(T_c\right)}_{MC}]$ with ${\left(T_c\right)}_{\mathrm{exact}}=\frac{2}{log(1+\sqrt{2})}=2.27$ (vertical gray line), and ${\left(T_c\right)}_{MC}=2.35$ (vertical light-yellow line) the theoretical and Monte Carlo critical temperatures, respectively.

Figure 7

(a) Color chart of $\mathrm{JSD}\left[P_{T_{\mathrm{flow}}}\right|\left|Q_T\right]$ of $T-T_{\mathrm{flow}}$ plane for the $q=4$ clock model on a $40\times 40$ lattice for the ensemble of NN-flow states after 100 iteration steps. The corresponding diagram of magnetic susceptibility (yellow line with decorated crosses) is also superimposed, which shows a peak near $T={\left(T_c\right)}_{\mathrm{exact}}$. (b) NN-flow $\mathrm{JSD}\left[P_{T_{\mathrm{flow}}}\right|\left|Q_T\right]$ for $T_{\mathrm{flow}}=T_{\mathrm{low}}$ (red), $T_{\mathrm{mid}}$ (blue), and $T_{\mathrm{high}}$ (green) of Fig. 5 on a $20\times 20$ lattice (dashed line) and $40\times 40$ lattice (solid line). We label ${\left(T_c\right)}_{\mathrm{exact}}=\frac{1}{log(1+\sqrt{2})}=1.13$ by the vertical gray dash-dot line, and ${\left(T_c\right)}_{MC}=1.24(20\times 20\mathrm{lattice}), 1.20(40\times 40\mathrm{lattice})$ with the vertical solid light-yellow lines. We see that the finite-size effect is relevant to improve the capability of using the minima of NN-flow JSDs to identify the critical temperature. Our results again show that NN flows can reach a fixed-point JSD pattern with its minimum located at the critical temperature.

Figure 8

NN-flow $\mathrm{JSD}\left[P_{T_{\mathrm{flow}}}\right|\left|Q_T\right]$ after 100 iterations for $T_{\mathrm{flow}}=T_{\mathrm{low}}$(red solid), $T_{\mathrm{mid}}$ (blue dashed), and $T_{\mathrm{high}}$ (green dotted) of Fig. 5 on a $40\times 40$ lattice for the clock models of $q=$ (a) 5, (b) 6, and (c) 7. The critical temperatures ${\left(T_1\right)}_{MC}$ (yellow solid) and s ${\left(T_1\right)}_{MC}$ (yellow dashed) of the Monte Carlo simulation are indicated by the vertical light-yellow lines in each subfipanel. The universal JSDs have decreased sharply near the transition point ${\left(T_1\right)}_{MC}$ between the ordered phase and the BKT phase. Moreover, the JSD minima sit right at $T={\left(T_2\right)}_{MC}$ in (a) and (b), but a bit higher than $T={\left(T_2\right)}_{MC}$ in (c).

Figure 9

(a) Color chart of $\mathrm{JSD}\left[P_{T_{\mathrm{flow}}}\right|\left|Q_T\right]$ on the $T-T_{\mathrm{flow}}$ plane with $P_{T_{\mathrm{flow}}}$ and $Q_T$ the PDFs of the mean spin value for the Monte Carlo ensembles of the $q=8$ clock model on a $40\times 40$ lattice at temperatures $T_{\mathrm{flow}}$ and $T$, respectively. The three (red) blocks coincide with the ordered, BKT, and disordered phases, which are superimposed by the magnetic susceptibility, shown by the light-yellow line with decorated crosses. (b) The JSD color chart with the PDF $P_{T_{\mathrm{flow}}}$ in (a) replaced by the NN-flow one after 100 iterations. The uniform color distribution along the $y$ axis indicates a universal fixed-point ensemble, which can capture the Monte Carlo critical temperatures ${\left(T_{1,2}\right)}_{MC}\approx 0.39,1.06$ by its discontinuity along the $x$ axis.

Figure 10

NN-flow $\mathrm{JSD}\left[P_{T_{\mathrm{flow}}}\right|\left|Q_T\right]$ after 100 iterations for $T_{\mathrm{flow}}=T_{\mathrm{low}}$ (red solid), $T_{\mathrm{mid}}$ (blue dashed), and $T_{\mathrm{high}}$ (green dotted) of Fig. 5 for the clock models of $q=8$ (a) on a $20\times 20$ lattice, and (b) on a $40\times 40$ lattice. The results show that NN flow can yield a universal JSD to identify the BKT phase and the associated critical temperatures by taking account of the finite-size effect.

Figure 11

Correlation of site spins, $\langle s_i{s}_j\rangle$ of the $q=8$ clock model on a $40\times 40$ lattice as a function of $r=|i-j|$ for the Monte Carlo ensemble at various temperatures and for the universal NN-flow ensemble after 100 iterations (dashed lines). The power-law behavior will be characterized by an approximately straight line of nonzero slope. The $T=0.8$ one (black square line) represents the BKT phase and does show an approximate power-law behavior. However, the NN-flow one is a flat straight line, more like the low-temperature ordered phase (red circle line).

Figure 12

(a) Picture of the orientations of the site spins for a chosen BKT Monte Carlo state of $q=8$ clock model on a $40\times 40$ lattice with temperature $T=0.8$. This picture shows the typical vortex-antivortex condensation of the BKT phase. (b) The picture of the orientations of the site spins for some six chosen NN-flow states of the same model after 20 iterations. For simplicity, only part of the lattice is shown. Almost all the site spins in each NN-flow state point to the same direction.

Figure 13

Probability density functions (PDF) for the mean spin value $M_n$ of $q=8$ clock model on $40\times 40$ lattice. Blue: the PDFs for the temperature ensembles of Monte Carlo simulated configurations at (a) $T=0.2$ (ordered phase), (b) $T=0.36$ [ordered phase, near ${\left(T_1\right)}_{MC}$], (c) $T=0.48$ [BKT phase, near ${\left(T_1\right)}_{MC}$], (d) $T=0.96$ [BKT phase, near ${\left(T_2\right)}_{MC}$], (e) $T=1.12$ [disordered phase, near ${\left(T_2\right)}_{MC}$], and (f) $T=1.6$ (disordered phase). Red: the PDFs of the corresponding NN-flow ensemble after 100 iterations, which are almost the same for all temperatures. The results show that although the flow PDFs do not resemble the ones at ${\left(T_2\right)}_{MC}$, it still yields the maximal overlap and thus the minimal JSD to help indicate the phase transition.

Figure 14

(a) Energy and (b) heat capacity for $q=2,3,4,5,6,7,8$ clock model is obtained from the Monte Carlo configurations on a $40\times 40$ lattice.

Figure 15

A typical schematic architecture of a variational autoencoder for training the unit of NN flow with $L=40$. The number inside the brackets indicates the dimension of each layer. In the middle are the hidden layers of VAE with dimensions indicated as 128 and 144, denoted as $h_1$ and $h_2$, respectively, in Fig. 15.

Figure 16

Typical histograms of training accuracy for the NN model with different parameters, i.e., the dimensions of two hidden layers of VAE, $h_1$ and $h_2$ in Fig. 15: (a) $h_1=128$ and $h_2=144$; (b) $h_1=128$ and $h_2=36$; (c) $h_1=36$ and $h_2=36$.

Figure 17

NN-flow $\mathrm{JSD}\left[P_{T_{\mathrm{flow}}}\right|\left|Q_T\right]$ of the $q=4$ clock model on a $40\times 40$ lattice for the ensemble of NN-flow states after 100 iteration steps with different hyperparameters. Our results again show that NN flows can reach a fixed-point JSD pattern with its minimum located at the critical temperature. The minimum points seem to be independent of the hyperparameter of NN.