Machine Learning Phases of Strongly Correlated Fermions

Machine learning offers an unprecedented perspective for the problem of classifying phases in condensed matter physics. We employ neural-network machine learning techniques to distinguish finite-temperature phases of the strongly correlated fermions on cubic lattices. We show that a three-dimensional convolutional network trained on auxiliary field configurations produced by quantum Monte Carlo simulations of the Hubbard model can correctly predict the magnetic phase diagram of the model at the average density of one (half filling). We then use the network, trained at half filling, to explore the trend in the transition temperature as the system is doped away from half filling. This transfer learning approach predicts that the instability to the magnetic phase extends to at least 5% doping in this region. Our results pave the way for other machine learning applications in correlated quantum many-body systems.

Popular Summary

Artificial neural networks—computing systems that operate on principles similar to neurons in the brain—can be trained to solve complex problems (such as handwriting recognition) that are difficult for traditional algorithms. These machine learning techniques can be applied to solving problems in quantum physics such as how to distinguish phases of matter. One example is quantum magnetism, where electrons at low temperatures rearrange themselves to order their intrinsic magnetic fields. Using conventional methods, physicists look for the onset of these rearrangements by studying physical observables, such as correlations between magnetic alignments at long distances, as model parameters are varied. Such classifications have interesting similarities to classifying images, which neural networks can do efficiently. Here, we show that it is possible for a machine to not only learn to pick up sudden changes in the collective magnetic properties of electrons but also explore situations that are difficult to study using conventional methods.

We find that a neural network can be trained to identify the finite-temperature magnetic phase transition of interacting electrons on cubic lattices and then predict the trend in the transition temperature as the strength of the interaction is tuned. The training and classifications are done on configurations generated through quantum Monte Carlo simulations of the Hubbard model (a simple model of interacting particles in a lattice) with an average density of one electron per site. We transfer the learning to cases where the density is incommensurate with the lattice and make predictions about the fate of the ordered phase.

Our results pave the way for the development of machine learning techniques to solve quantum many-body systems.

Figure 1

Architecture of the 3D convolutional neural network used to obtain $T_N$ for the 3D Hubbard model. For input, we use the auxiliary field configurations in a four-dimensional grid, three spatial dimensions of size 4 (total of $N=4^3$ sites), and one imaginary time dimension of size $\mathcal{L}=200$. Numbers of volumetric feature maps in the hidden-feature extraction layers are $n^{(2)}=32$, $n^{(3)}=16$, and $n^{(4)}=8$. Here, $n^{(5)}=8$ in the fully connected layer. During training, dropout regularization with a rate of 0.5 was used to mitigate overtraining. To classify the input system as ordered or unordered, each of the eight fully connected neurons is connected to each of the two readout neurons using the softmax function as a neural activation function. The output neuron with the highest probability represents the activated neuron.

Figure 2

Prediction of the Néel transition temperature by the neural network. Using the auxiliary spin configurations, the network is trained separately at $U=5$ and $U=16$ for $N=4^3$, and simultaneously at $U=5$ and 16 for $N=4^3$ and $N=8^3$. The error bars are the standard error of the mean of six different classifications using CNNs that were trained starting from different random weights and biases. The critical temperatures used for the training of the network with $N=4^3$ are shown as stars (see text). Grey filled symbols are the estimates for $T_N$ in the thermodynamic limit from DQMC and NLCE simulations. Grey pentagons, hexagons, and circles for weak-, intermediate-, and strong-coupling regimes are taken from Refs. [18, 19, 21], respectively. The solid line is a guide to the eye.

Figure 3

Average output of the neuron that is trained to be activated (return 1) in the ordered phase as a function of temperature at half filling. The network is maximally confused (average is 0.5) at the transition temperatures. We use 90 000 configurations for $N=4^3$ and 10 000 configurations for $N=8^3$. The grey solid lines are fits to data in the ranges shown. The vertical dashed lines indicate the estimates for $T_N$ in the thermodynamic limit (about 0.30 for both $U$ values) from other studies (see caption of Fig. 2).

Figure 4

Phase transitions away from half filling. (a) Magnetic phase diagram of the 3D Hubbard model away from half filling for $U=9$. Empty red (filled grey) symbol(s) are estimates for the Néel temperatures, taking (not taking) the minus sign problem into account, obtained from a neural network that is trained to identify the phase at half filling. Lines are guides to the eye. Inset: Equations of state (density vs $\mu$) for $U=9$ at $T=0.24$, 0.28, and 0.32 (solid lines from top to bottom, respectively). The grey dashed line is calculated without taking the sign problem into account at $T=0.32$. (b) Average neuron output $\mathcal{O}$ calculated taking (without taking) the sign into account at $T=0.32$ ($⟪\mathcal{SO}⟫/⟪\mathcal{S}⟫$ and $⟪\mathcal{O}⟫$, respectively). See Appendix pp1 for details. The grey dotted line is a fit to the data near the 0.5 crossing point. (c) Same as in panel (b) but for $T=0.30$ and 0.28.

Figure 5

Construction of a volumetric feature map. To produce the highlighted neuron in the volumetric feature map, from the input, a shared filter ${\mathit{w}}_{l1}^{(2)}$ is convolved with a $2\times 2\times 2$ local receptive cube ${\mathit{x}}_l^{\prime }$ across all the inputs in the imaginary time slices, adding a bias offset $b_1^{(2)}$, before being activated using ReLU. Sliding the $2\times 2\times 2$ cube in each of the spatial dimension while repeating the convolutional procedure completes a volumetric feature map. The spatial size of the volumetric feature map can be preserved by using zero padding.

Figure 6

Progress of the training process. Training accuracy for (a) simultaneous training with $U=5$ and 16, (b) $U=5$ alone, and (c) $U=16$ alone vs epochs. The location of last saved model is shown as a green circle in each panel. The testing accuracies for saved models are 92.7% (93.2%) for $N=4^3$ ($N=8^3$) in (a), 91.9% in (b), and 94.3% in (c).

Figure 7

(a)–(d) Typical auxiliary field patterns above and below $T_N$ used as input to the CNN for the $N=4^3$ system. In each panel, the four rows correspond to the four layers in the $z$ direction. We show the field only in the first 10 imaginary time slices (left to right) for each case. (e,f) Normalized count of the 256 patterns within $2\times 2\times 2$ cubes as detected in the raw configurations for $U=16$ below and above $T_N$. The two largest peaks in (f) correspond to the two AFM patterns.

Figure 8

Histogram of the overlap between the learned features and all the possible ordering patterns picked up by the $2\times 2\times 2$ receptive cube. Top and bottom panels correspond to cases with $U=5$ and $U=16$, respectively. The vertical dashed (red) line marks the value for the perfect AFM ordering pattern.