Group-equivariant autoencoder for identifying spontaneously broken symmetries

We introduce the group-equivariant autoencoder (GE autoencoder), a deep neural network (DNN) method that locates phase boundaries by determining which symmetries of the Hamiltonian have spontaneously broken at each temperature. We use group theory to deduce which symmetries of the system remain intact in all phases, and then use this information to constrain the parameters of the GE autoencoder such that the encoder learns an order parameter invariant to these “never-broken” symmetries. This procedure produces a dramatic reduction in the number of free parameters such that the GE-autoencoder size is independent of the system size. We include symmetry regularization terms in the loss function of the GE autoencoder so that the learned order parameter is also equivariant to the remaining symmetries of the system. By examining the group representation by which the learned order parameter transforms, we are then able to extract information about the associated spontaneous symmetry breaking. We test the GE autoencoder on the 2D classical ferromagnetic and antiferromagnetic Ising models, finding that the GE autoencoder (1) accurately determines which symmetries have spontaneously broken at each temperature; (2) estimates the critical temperature in the thermodynamic limit with greater accuracy, robustness, and time efficiency than a symmetry-agnostic baseline autoencoder; and (3) detects the presence of an external symmetry-breaking magnetic field with greater sensitivity than the baseline method. Finally, we describe various key implementation details, including a quadratic-programming-based method for extracting the critical temperature estimate from trained autoencoders and calculations of the DNN initialization and learning rate settings required for fair model comparisons.

Figure 1

(a) Generators of the spatial Ising symmetries acting on a square lattice and (b) some example compound spatial symmetries that can be formed from the generators.

Figure 2

Schematic illustration of the autoencoder architecture.

Figure 3

(a) Schematic illustration of an encoder constrained to be invariant to a group of symmetries (even-unit cyclic translations in the illustration). Network edges of same line style (solid vs dashed vs dotted) and shade are constrained to have equal weight. (b) Reduced representation of the same encoder. Rather than replicating weights in the first layer, lattice sites that feed into the same input neuron are first averaged.

Figure 4

Distributions of the order observable values at each temperature $T$ on a lattice of size $L=128$ for one example training data fold and initialization seed and using $N=256$ training-validation samples per temperature. The top row shows results for the ferromagnetic model with the observable derived from (a) the magnetization, (b) the baseline AE, and (c) the GE AE. The bottom row (d)–(f) shows corresponding results obtained for the antiferromagnetic model. The observable scales ($y$ axes) of the baseline encoder and GE encoder are arbitrary, as any nonzero rescaling of the encoder can be compensated by the inverse scaling in the decoder; thus, only the relative shapes of these distributions are meaningful. All three order observables suggest a phase transition in both the ferromagnetic and antiferromagnetic cases near the theoretical critical temperature (dashed red).

Figure 5

Order parameters (mean absolute value of the distributions in Fig. 4) vs temperature $T$ on a lattice of size $L=128$ for one example training data fold and initialization seed and using $N=256$ training-validation samples per temperature. The top row shows results for the ferromagnetic model with the observable derived from (a) the magnetization, (b) the baseline AE, and (c) the GE AE. The bottom row (d)–(f) shows corresponding results obtained for the antiferromagnetic model. The curves have been normalized to the same scale as Onsager's solution for comparison. The standard deviation curves (dashed) were obtained using the jackknife resampling method (these may be difficult to see as the standard deviations are small). All three order parameters suggest a phase transition in both the ferromagnetic and antiferromagnetic cases near the theoretical critical temperature (dashed red). However, the order parameters derived from magnetization and the GE encoder are smoother and give better approximations to Onsager's solution (dashed blue).

Figure 6

Fourth-order Binder cumulants $U_4$ of the order observables at each temperature $T$ on a lattice of size $L=128$ for one example training data fold and initialization seed and using $N=256$ training-validation samples per temperature. The top row shows results for the ferromagnetic model with the data derived from (a) the magnetization, (b) the baseline AE, and (c) the GE AE. The bottom row (d)–(f) shows corresponding results obtained for the antiferromagnetic model. The standard deviation curves (dashed) were obtained using the jackknife resampling method (these may be difficult to see since the standard deviations are small). In all cases, the Binder cumulant drops abruptly near the theoretical critical temperature (dashed red). Step functions (blue) are fit to the Binder cumulants using least squares, and the locations of their jump discontinuities are taken as estimates of the critical temperature.

Figure 7

Ferromagnetic critical temperature estimates (expressed as percent errors relative to the exact theoretical critical temperature) for lattice sizes (a) $L=16$, (b) 32, (c) 64, and (d) 128. The error bars represent standard deviations that combine the standard deviation of means across 24 trials (eight training data folds and three initialization seeds) and the jackknife standard deviations for each fold and seed, with most of the variance coming from the former. The standard deviation associated with magnetization is 0. The baseline encoder consistently achieves the lowest error and improves with more training data. The GE encoders, on the other hand, are more similar to magnetization, with multiscale training reducing error slightly, and are also more stable in terms of their standard deviations.

Figure 8

Antiferromagnetic critical temperature estimates (expressed as percent errors relative to the exact theoretical critical temperature) for lattice sizes (a) $L=16$, (b) 32, (c) 64, and (d) 128. The error bars represent standard deviations that combine the standard deviation of means across 24 trials (eight training data folds and three initialization seeds) and the jackknife standard deviations for each fold and seed, with most of the variance coming from the former. The standard deviation associated to magnetization is 0. The baseline encoder consistently achieves the lowest error and improves with more training data. The GE encoders, on the other hand, are more similar to magnetization, with multiscale training reducing error slightly, and are also more stable in terms of their standard deviations.

Figure 9

Linear dependence of critical temperature estimates (expressed as percent errors relative to the exact theoretical critical temperature) on inverse lattice size using $N=256$ training-validation samples per temperature and averaged over 24 trials (eight training data folds and three initialization seeds). Results for the ferromagnetic model are shown in (a), and results for the antiferromagnetic model are shown in (b). Although the baseline encoder achieves lower error for each lattice size individually, the linear extrapolation of its estimates to infinite lattice size ($y$ intercept) is a worse estimate compared to the GE encoders.

Figure 10

Critical temperature estimates (expressed as percent errors relative to the exact theoretical critical temperature) extrapolated to infinite lattice size. Results for the ferromagnetic (FM) model are shown in panel (a), and results for the antiferromagnetic (AFM) model are shown in panel (b). The error bars represent standard deviations that combine the standard deviation of means across 24 trials (eight training data folds and three initialization seeds) and the jackknife standard deviations for each fold and seed, with most of the variance coming from the former. The standard deviation associated to magnetization is 0. Although the baseline encoder achieves lower error on finite lattices (Figs. 7 and 8), the GE encoder estimates extrapolate to have lower error in the thermodynamic limit.

Figure 11

Computation times to generate data (black) and train (not black) the autoencoders for each lattice size $L$. Results are shown for the (a) ferromagnetic and (b) antiferromagnetic cases. The time to measure magnetization is reported as equivalent to data generation time as these measurements were taken on the fly during the MC simulation. Each training time is a sum over all eight training folds and RNG seeds. The training times were then averaged over all training-validation sample sizes $N$ and over both the ferromagnetic and antiferromagnetic cases, as these factors did not contribute to significant variation in execution time. The GE autoencoder is significantly more time-efficient (lower time) than the baseline autoencoder. The times reported for infinite lattice size are just sums of the times for the finite lattice sizes, although the multiscale GE autoencoder exhibits greater efficiency as it only requires a single network to be trained across all finite lattice sizes.

Figure 12

“Confidence scores” $\xi$ [see Eq. (17)] of the baseline encoder (white) and GE encoder (dark gray) at detecting the presence of an external magnetic field at temperatures (a) $T=2<T_c$ and (b) $T=2.5>T_c$. Results are shown for three different field strengths $h$. The error bars represent standard deviations estimated using $10000$ bootstrap draws from the 24 trials used to calculate $\xi$. The GE encoder consistently detects the external field with greater confidence than does the baseline encoder.

Figure 13

Measure of error between the baseline encoder and magnetization applied to the (a) ferromagnetic (FM) and (b) antiferromagnetic (AFM) models. Error bars indicate the standard deviation across 24 trials (eight training data folds and three initialization seeds). The baseline encoder is distinct from magnetization (positive error $\nu$) and becomes increasingly distinct with increasing number of training-validation samples in the ferromagnetic case.

Figure 14

Measure of error between the order parameters derived from each order observable and Onsager's solution using $N=256$ training-validation samples per temperature and averaged over 24 trials (eight training data folds and three initialization seeds). Results for the ferromagnetic model are shown in panel (a), and results for the antiferromagnetic model are shown in panel (b). The error exhibits an approximately linear dependence on inverse lattice size, with stronger linearity for magnetization and the GE encoders. At infinite lattice size ($L^{-1}=0$), magnetization and the GE encoders converge more closely to Onsager's solution than does the baseline encoder.