Algorithms for tensor network renormalization

We discuss in detail algorithms for implementing tensor network renormalization (TNR) for the study of classical statistical and quantum many-body systems. First, we recall established techniques for how the partition function of a $2D$ classical many-body system or the Euclidean path integral of a $1D$ quantum system can be represented as a network of tensors, before describing how TNR can be implemented to efficiently contract the network via a sequence of coarse-graining transformations. The efficacy of the TNR approach is then benchmarked for the $2D$ classical statistical and $1D$ quantum Ising models; in particular the ability of TNR to maintain a high level of accuracy over sustained coarse-graining transformations, even at a critical point, is demonstrated.

Figure 1

(a) A square lattice of classical spins $\sigma \in \{+1,-1\}$. (b) The partition function of the classical system can be encoded as a square network (tilted ${45}^{\circ }$ with respect to the spin lattice) of four-index tensors $A_{ijkl}$, with a tensor sitting in the center of every second plaquette of spins. Here each tensor $A$ encodes the Boltzmann weights associated with the interactions of spins on the edges of the plaquette; see Eq. (3).

Figure 2

(a) The imaginary time-evolution operator $e^{-\beta H}$ is expressed via Suzuki-Trotter expansion as a product of two-site gates $e^{-\tau h}$. Red dashed lines denote how the gates are to be decomposed at the next step. (b) The two-site gates $e^{-\tau h}$ are decomposed into a product of ternary tensors according to either a horizontal partition (accomplished via singular value decomposition) or a vertical partition (accomplished via eigen-decomposition). (c) Groups of four ternary tensors are contracted together to form four-index tensors $A$. (d) Depictions of the vertical and horizontal partitions of the two-site gates $e^{-\tau h}$, and definition of the four index tensors $A$.

Figure 3

(a) Depiction of (the square of) the Hilbert-Schmidt norm of a four-index tensor $u$. Note that a darker shade is used to represent the conjugate tensor, which is also drawn with opposite vertical orientation. (b) Given a square lattice tensor network, we wish to replace a $2\times 2$ block of tensors $\mathcal{F}$ from the network with a different subnetwork of tensors $\tilde{\mathcal{F}}$. (c) The square of the difference between $\mathcal{F}$ and $\tilde{\mathcal{F}}$ under the Hilbert-Schmidt norm is depicted, where darker shades are used to depict conjugate tensors, which are drawn with opposite vertical orientation to regular tensors. The replacement in (b) is valid if the difference $\parallel \mathcal{F}-\tilde{\mathcal{F}}\parallel$ is sufficiently small. (d) Assuming that the local square-lattice network is homogeneous, one can replace $\mathcal{F}$ with $\tilde{\mathcal{F}}$ in all $2\times 2$ blocks. A coarser square-lattice network is obtained after contraction between pairs of three-index tensors.

Figure 4

(a) In a projective truncation a subnetwork $\mathcal{F}$ is replaced by a new subnetwork $\tilde{\mathcal{F}}$, which consists of a projector $P$ applied to the original subnetwork, i.e., $\tilde{\mathcal{F}}=\mathcal{F}P$. (b) Here we assume that $P$ is decomposed as a product of an isometric tensor $w$ and its conjugate, $P=w{w}^{†}$. (c) By definition, isometry $w$ contracts to identity with its conjugate, $w^{†}w=\mathbb{I}$. (d) The square of the error in a projective truncation is expanded as a sum of four terms; however given that $P^2=P$, two of the terms cancel; see also Eq. (18). (e) The environment ${\mathrm{\Gamma }}_w$ of isometry $w$ is defined as the network that results by removing a single instance of $w$ from ${‖\mathcal{F}w‖}^2$; see also Eq. (20). (f) By construction, the contraction of $w$ and its environment ${\mathrm{\Gamma }}_w$ is equal to ${‖\mathcal{F}w‖}^2$. (g) Environment ${\mathrm{\Gamma }}_w$ is decomposed, via singular value decomposition (SVD), into a product of isometric tensors $u, v$, and diagonal matrix $s$.

Figure 5

The sequence of coarse-graining steps used in the binary TNR scheme in order to map an initial square lattice of tensors $A$, where every second row of tensors has been conjugated as described in Appendix pp4, to a coarser square lattice composed of tensors $A^{\prime }$. (a) A projective truncation is made on all $2\times 2$ blocks of tensors; see Figs. 6–6. (b) Conjugate pairs of disentanglers $u$ are contracted to identity. (c) A projective truncation is made on all $B$ tensors; see Figs. 6–6. (d) A final projective truncation is made; see Figs. 6 and 6 for details. (e) Conjugate pairs of isometries $w$ are contracted to identity.

Figure 6

(a) Details of the projective truncation made at the first step of the TNR iteration; here two copies of a projector $P_u$, which is composed of a product isometric and unitary tensors, are applied to a $2\times 2$ block of $A$ tensors. (b) Definition of four-index tensor $B$. (c) Projector $P_u$ is formed from isometries $v_L, v_R$ and disentangler $u$ (and their conjugates). (d) Details of the projective truncation made at the second step of the TNR iteration. (e) Definition of matrix $D$. (f) Details of the projective truncation made at the third step of the TNR iteration. (g) Definition of new four-index tensor $A^{\prime }$, copies of which comprise the coarse-grained square-lattice tensor network. (h) Delineation of the different dimensions $\{{\chi }_u,{\chi }_v,{\chi }_w,{\chi }_y\}$ of indices on tensors $\{u,v_L,v_R,y_L,y_R,w\}$.

Figure 7

The linearized environments of tensors $\{v_L,v_R,u,y_L,y_R,w\}$ involved in an iteration of the binary TNR scheme. (a)–(c) Environments ${\mathrm{\Gamma }}_{v_L}, {\mathrm{\Gamma }}_{v_R}$, and ${\mathrm{\Gamma }}_u$ of the isometries $v_L, v_R$ and disentangler $u$ involved in the first projective truncation of the TNR iteration, as detailed in Fig. 6. (d) and (e) Environments ${\mathrm{\Gamma }}_{y_L}$ and ${\mathrm{\Gamma }}_{y_R}$ of isometries $y_L$ and $y_R$ from the second projective truncation of the TNR iteration, as detailed in Fig. 6. (f) Environment ${\mathrm{\Gamma }}_w$ of isometry $w$ from the third projective truncation of the TNR iteration, as detailed in Fig. 6.

Figure 8

(a)–(e) An iteration of the TNR coarse-graining transformation for a square-lattice tensor network with an impurity tensor $M^{\left(0\right)}$. The projective truncations of the TNR iteration are applied as usual, see Fig. 5, neglecting those which are incompatible impurity tensor. (f) The coarse-grained square lattice is homogeneous everywhere except for a $2\times 2$ region occupied by new impurity tensor $M^{\left(1\right)}$. (g) The coarse-grained impurity tensor $M^{(s+1)}$ is given by enacting ascending superoperator $\mathcal{R}$, defined from two-body gates $\{G_R,G_L,G_U,G_Y\}$, on the impurity tensor $M^{\left(s\right)}$. (h) Definition of two-body gates $\{G_R,G_L,G_U,G_Y\}$.

Figure 9

A finite network with periodic boundaries and a local impurity $M^{\left(s\right)}$ is coarse-grained into a single-impurity tensor $M^{(s+1)}$ under the transformation depicted in Fig. 8. The expectation value of the local observable corresponding to the impurity is given from the trace of $M^{(s+1)}$ as depicted.

Figure 10

(a) An $8\times 8$ square lattice of tensors that has had a $2\times 2$ block of tensors removed, leaving 8 open indices. (b) The lattice after one iteration of coarse-graining with TNR. (c) The lattice after a second iteration of coarse-graining with TNR. (d) The network from (c) is redrawn on the cylinder. Each double layer of the network corresponds to the ascending superoperator $\mathcal{R}$ as defined in Fig. 8. (e) An $8\times 8$ square lattice of tensors with an impurity $\sigma$ positioned on a link. (f) The network from (e) after two iterations of coarse-graining with TNR.

Figure 11

(a) Comparison between TRG and TNR of the truncation error $\varepsilon$, as defined in Eq. (11), as a function of RG step $s$ in the $2D$ classical Ising model at critical temperature $T_c$. While increasing the bond dimension $\chi$ gives smaller truncation errors, the truncation errors still grow quickly as a function of RG step $s$ under TRG. Conversely, truncation errors remain stable under coarse-graining with TNR. (b) Relative error in the free energy per site $\delta f$ at the critical temperature $T_c$, comparing TRG and TNR over a range of bond dimensions $\chi$. The error from TRG is seen to diminish polynomially with bond dimension, with fit $\delta f\propto {\chi }^{-3.02}$ (where the inset displays the same TRG data with logarithmic scales on both axes), while the error from TNR diminishes exponentially with bond dimension, with fit $\delta f\propto exp(-0.305\chi )$. Extrapolation suggests that TRG would need bond dimension $\chi \approx 750$ to match the accuracy of the $\chi =42$ TNR result.

Figure 12

(a) Spontaneous magnetization $M\left(T\right)$ of the $2D$ classical Ising model near critical temperature $T_c$, both exact and obtained with TNR with $\chi =6$. Even very close to the critical temperature, $T=0.9994T_c$, the magnetization $M\approx 0.48$ is reproduced to within $1\%$ accuracy. (b) Specific heat, $c\left(T\right)=-T\frac{{\partial }^{2}f}{\partial T^2}$, both exact and obtained using TNR with $\chi =6$.

Figure 13

The precision with which TNR approximates a scale-invariant fixed-point tensor for the $2D$ classical Ising model at critical temperature $T_c$ is examined by comparing the difference between tensors produced by successive TNR iterations ${\delta }^{\left(s\right)}\equiv \parallel A^{\left(s\right)}-A^{(s-1)}\parallel$, where tensors have been normalized such that $\parallel A^{\left(s\right)}\parallel =1$. The precision with which scale invariance is approximated in the initial RG steps (small $s$) is limited by the presence of RG-irrelevant terms in the lattice Hamiltonian that break scale invariance at short-distance scales, while numerical truncation errors, which can be thought of as introducing RG-relevant terms, shift the system from criticality (and thus scale invariance) in the limit of many RG steps $s$.

Figure 14

The smallest 101 scaling dimensions of the $2D$ classical Ising model at critical temperature $T_c$, obtained by diagonalizing the ascending superoperator $\mathcal{R}$, see Fig. 8, using TNR with bond dimension $\chi =6$. The scaling dimensions are organized according to the parity $p=\pm 1$ (even/odd) and locality of the corresponding scaling operators. All scaling dimensions shown are reproduced to within $2\%$ of their exact values.

Figure 15

(a) Relative error in the energy of scale-invariant MERAs optimized for the ground state of the $1D$ quantum Ising model at criticality in terms of bond dimension $\chi$, comparing MERAs optimized using TNR to those optimized using variational energy minimization. Energy minimization produces MERAs with a more accurate approximation to the ground state energy, but is significantly more computationally expensive [with a computational cost that scales as $O\left({\chi }^9\right)$ versus as $O\left({\chi }^6\right)$ for TNR]. (b) Low-energy eigenvalues of the $1D$ quantum Ising model at criticality as a function of $1/L$, computed with $\chi =12$ TNR. Discontinuous lines correspond to the finite-size CFT prediction, which ignores corrections of order $L^{-2}$.

Figure 16

(a) Thermal energy per site (above the ground state energy) as a function of the inverse temperature $\beta$, for the $1D$ quantum Ising model in an infinite chain, for different values of magnetic field $\lambda$. Data points are computed with $\chi =12$ TNR while continuous lines correspond to the exact solution. (b) Connected two-point correlators at the critical magnetic field $\lambda =1$, as a function of the distance $d$, for several values of $\beta$. Data points are computed with $\chi =12$ TNR while continuous lines again correspond to the exact solution.

Figure 17

An iteration of a coarse-graining scheme that acts to compress only the vertical dimension of the square-lattice tensor network. (a) A projective truncation, involving isometries $w$, is employed. (b) Details of the projective truncation from (a); here a projector $P$, composed of an isometry $w$ and its conjugate, is applied to a pair of $A$ tensors. (c) The coarse-grained network, composed of tensors $A^{\prime }$, is given via tensor contractions. (d) Definition of the coarse-grained tensor $A^{\prime }$.

Figure 18

A depicted of an iteration of the ternary TNR scheme, which maps a square-lattice tensor network to a coarse-grained square lattice of one-third the linear dimension of the original. (a) A projective truncation, detailed in Fig. 19. (b) Pairs of disentanglers $u$ annihilate to identity. (c) A projective truncation, detailed in Fig. 19. (d) A projective truncation, detailed in Fig. 19. (e) Pairs of isometries $w$ annihilate to identity, yielding the coarse-grained square lattice network.

Figure 19

Overview of the projective truncations involved in the ternary TNR scheme. (a) A projective truncation, implemented by projector $P_u$ composed of isometries $v_L, v_R$ and disentangler $u$, acts upon a $3\times 2$ block of $A$ tensors. (b) A projective truncation, formed from isometries $y_L, y_R$ and their conjugates, acts upon $B$ tensors. (c) A projective truncation, formed from isometry $w$ and its conjugate, acts to give the new four-index tensor $A^{\prime }$ of the coarse-grained network.

Figure 20

Depiction of a single iteration of the isotropic TNR scheme, which maps a square-lattice network, composed of tensors $A_b, A_p$, and $A_r$, to a coarser lattice of the same type. (a) Two types of projective truncation are made, as detailed in Figs. 21 and 21. (b) Conjugate pairs of disentanglers $u$ annihilate to identity. (c) Two types of projective truncation are made, as detailed in Figs. 21 and 21. (d) Conjugate pairs of isometries $w$ annihilate to identity, yielding a coarse-grained network composed of tensors ${A^{\prime }}_b, {A^{\prime }}_p$, and ${A^{\prime }}_r$.

Figure 21

Overview of the projective truncations involved in an iteration of the isotropic TNR scheme. (a) A projective truncation, involving isometries $v$ and disentanglers $u$, is enacted upon a product of four $A_p$ tensors and a $A_b$ tensor. The coarse-grained tensor ${A^{\prime }}_b$ is also defined. (b) A projective truncation, involving isometries $y$ and disentanglers $u$, is enacted upon $A_r$ tensors. (c) A projective truncation, involving isometries $w$, is enacted, yielding coarse-grained tensors ${A^{\prime }}_r$. (d) A final projective truncation, again involving isometries $w$, is applied to a product of $A_b$ and $v$ tensors, yielding coarse-grained tensors ${A^{\prime }}_p$.

Figure 22

(a) The tensor $A$ is Hermitian symmetric if a row of such tensors is invariant with respect to permutation of top-bottom indices in conjunction with complex conjugation. (b) Permutation and complex conjugation of Hermitian symmetric $A$ is equivalent to a gauge change, enacted by some unitary matrix $\mathbf{x}$, on its horizontal indices. (c) Definition of $A^{†}$. (d) A gauge change is performed on the tensors of every second row of the square-lattice network. (e) Definition of tensor $Q$. (f) Tensor $Q$ is seen to be Hermitian. (g) and (h) Constraints for isometries $y_L$ and $y_R$ necessary such that the tensors of the coarse-grained network remain Hermitian symmetric.

Figure 23

(a) Details of an optional, preliminary projective truncation for the binary TNR scheme where two copies of a projector $P$, which is formed from a product isometric tensors $q_L, q_R, z$ and their conjugates, are enacted upon a $2\times 2$ block of $A$ tensors. (b) Definition of tensor $K$. (c) Definition of tensor $\overline{B}$, which differs from the previous $B$, see Fig. 6, only by an amount related to the truncation error of $P$ in (a). (d) Detail of the environment ${\mathrm{\Gamma }}_u$ computed from $\overline{B}$. The cost of contracting this network scales as $O\left({\chi }^6\right)$ in terms of the bond dimension $\chi$, as opposed to $O\left({\chi }^7\right)$ for the previous environment of Fig. 7.

Figure 24

(a) Isometric tensors $y_L, y_R, w$ yielding a more accurate coarse-graining transformation can found by optimizing them to minimize the truncation error when applied to a region of the network, here containing two copies of the $B$ tensor, that is larger than was previously considered in Figs. 6 and 6. Notice that minimizing the truncation error is equivalent to maximizing the norm of $\tilde{A}$. (b) Definition of tensor $\tilde{A}$. (c) The two contributions to the modified environment ${\tilde{\mathrm{\Gamma }}}_u$ for disentangler $u$, as generated from $\parallel \tilde{A}{\parallel }^2$.