Automatic differentiation for second renormalization of tensor networks

Tensor renormalization group (TRG) constitutes an important methodology for accurate simulations of strongly correlated lattice models. Facilitated by the automatic differentiation technique widely used in deep learning, we propose a uniform framework of differentiable TRG ($\partial \mathrm{TRG}$) that can be applied to improve various TRG methods, in an automatic fashion. $\partial \mathrm{TRG}$ systematically extends the essential concept of second renormalization [Phys. Rev. Lett. 103, 160601 (2009)] where the tensor environment is computed recursively in the backward iteration. Given the forward TRG process, $\partial \mathrm{TRG}$ automatically finds the gradient of local tensors through backpropagation, with which one can deeply “train” the tensor networks. We benchmark $\partial \mathrm{TRG}$ in solving the square-lattice Ising model, and we demonstrate its power by simulating one- and two-dimensional quantum systems at finite temperature. The global optimization as well as GPU acceleration renders $\partial \mathrm{TRG}$ a highly efficient and accurate many-body computation approach.

Figure 1

Part (a) shows a TRG step where the scale transformations $w$ are introduced along both directions, while only vertical renormalization is involved in (b), which eventually compresses the tensor network into a 1D structure that can then be contracted exactly. Part (c) plots the computational graph of the forward TRG process as well as the backpropagation.

Figure 2

The relative errors of free energy $|\delta f/f|$ in the vicinity of critical temperature $T_c$, obtained from HOTRG, HOSRG, and $\partial \mathrm{TRG}$ calculations. We perform $n_i=5$–10 iterations within each MERA update, and the total number of sweeps $n_s$ ranges from a few iterations to a few hundred, depending on the specific temperature and scheme, until the relative errors reach a convergence criterion of $\epsilon ={10}^{-4}$. The $D=24 \partial \mathrm{TRG}$ calculations follow the scheme in Fig. 1, while the $D=64$ and 128 cases follow the scheme in Fig. 1.

Figure 3

(a) Comparisons of relative errors of free energy between linearized TRG (LTRG) and $\partial \mathrm{TRG}$. In the initial $\rho \left(\tau \right)$ of $\partial \mathrm{TRG}, \tau \simeq 5\times {10}^{-5}$, which is also used as the Trotter slice in the LTRG calculations. The results are shown with optimization depth $n_d\le 4, n_i=10$ in a single MERA update, and overall sweep iterations $n_{\mathrm{s}}=3$. (b) Comparisons of the computational wall time of $\partial \mathrm{TRG}$ on GPU and CPU with up to 16 cores, benchmarked on the infinite $XY$ chain. The calculations are carried out on Nvidia Tesla V100 GPU and Intel Xeon 6230 CPU, with retained bond dimensions up to $D=360$. The dashed line depicts the scaling $t_h\sim D^4$.

Figure 4

(a) Relative errors of the transverse-field Ising model on a $4\times 4$ open square lattice, at critical transverse field $h=h_c$. (b) $\partial \mathrm{TRG}$ results with various $D$ are plotted vs optimization depths $n_d=1$, 2, 3, and 4. The comparison is at a fixed low temperature $\beta \simeq 105$, and the standard XTRG results are shown with solid lines.

Figure 5

The plot shows internal energy $u\left(T\right)$ and spin-specific heat $c_m\left(T\right)$ of the transverse field Ising model, with $h_x=1$ [(a), (b)] and $h_x=1.5$ [(c), (d)]. $\partial \mathrm{TRG}$ calculations are run on cylinders of various widths $W$ (with length $L$ extrapolated to infinity). The energy data show excellent agreements with the quantum Monte Carlo (QMC) result [52] on a $30\times 30\mathrm{square}$ lattice in both fields. For the $h_x=1$ case, the peak position in $c_m\left(T\right)$ provides an accurate estimate, $\sim 1\%$ relative error, of critical temperature.