Deep Autoregressive Models for the Efficient Variational Simulation of Many-Body Quantum Systems

Artificial neural networks were recently shown to be an efficient representation of highly entangled many-body quantum states. In practical applications, neural-network states inherit numerical schemes used in variational Monte Carlo method, most notably the use of Markov-chain Monte Carlo (MCMC) sampling to estimate quantum expectations. The local stochastic sampling in MCMC caps the potential advantages of neural networks in two ways: (i) Its intrinsic computational cost sets stringent practical limits on the width and depth of the networks, and therefore limits their expressive capacity; (ii) its difficulty in generating precise and uncorrelated samples can result in estimations of observables that are very far from their true value. Inspired by the state-of-the-art generative models used in machine learning, we propose a specialized neural-network architecture that supports efficient and exact sampling, completely circumventing the need for Markov-chain sampling. We demonstrate our approach for two-dimensional interacting spin models, showcasing the ability to obtain accurate results on larger system sizes than those currently accessible to neural-network quantum states.

Figure 1

Neural autoregressive quantum states are neural networks that represent a normalized wave function, $\mathrm{\Psi }(s_1,\dots ,s_N)$, by factoring it to a sequence of normalized conditional wave functions, denoted by ${\mathrm{\Psi }}_i(s_i|s_{i-1},\dots ,s_1)$ for the $i$th particle, in a manner similar to that of a neural autoregressive density estimator [see Eq. (3)]. (a) Illustration of a deep one-dimensional-convolutional NAQS model following the PixelCNN [18] architecture. Each column of nodes represents a layer in the network, starting with the input layer representing the $N$-particle configuration $(s_1,\dots ,s_N)$. Each internal node in the graph is a complex vector computed according to its layer type. Namely, masked convolutions are limited to having local connectivity, where a node at the $j$th row is only connected to nodes with connections to $s_i$ where $i<j$. All inputs to a node at the $l$th layer are multiplied by a matrix $W^{(l)}$, shared across all rows in the same layer, and followed by applying a nonlinear elementwise function $\sigma$: $\mathbb{C}\to \mathbb{C}$. (b) Depicts the exact sampling algorithm for NAQS, where empty nodes represent unused nodes, and filled but faded nodes represent cached results from previous steps. The quantum number of each particle is generated sequentially, by computing its respective conditional wave function, and sampling according to the squared magnitude. Notice that only a single row is processed at each step, and so sampling a complete configuration has the same runtime as a single forward pass.

Figure 2

An illustration of the two modes of the ground state, by taking the first two principal components of the generated samples. The green points correspond to our direct sampling method, and the other colors represent different MCMC chains. The plot was generated by training a NAQS on the transverse-field Ising model with $\mathrm{\Gamma }=2J$, below the critical value, on a $12\times 12$ lattice until convergence to the ground state, and then sampling from the trained NAQS using either our direct sampling method, or four separate MCMC samplers.

Figure 3

Comparing the effects of the sampling method, either MCMC or direct sampling, on the training procedure for the transverse-field Ising model with $\mathrm{\Gamma }=3J$, close to the critical value, on a large ($21\times 21$) lattice. When using MCMC, samples are taken every $k\in \{10,50,100,300\}$ steps in the chain, where increasing $k$ reduces the correlation between samples at the expense of increased computational cost. The solid lines shows the relative error to the minimal energy found for this system in our experiments, and dashed lines shows the energy variance. Since MCMC takes a considerable time to complete just a single iteration, we have restricted the training to a maximum of 100 hours.