Abstract
Differentiable programming is a fresh programming paradigm which composes parameterized algorithmic components and optimizes them using gradient search. The concept emerges from deep learning but is not limited to training neural networks. We present the theory and practice of programming tensor network algorithms in a fully differentiable way. By formulating the tensor network algorithm as a computation graph, one can compute higher-order derivatives of the program accurately and efficiently using automatic differentiation. We present essential techniques to differentiate through the tensor networks contraction algorithms, including numerical stable differentiation for tensor decompositions and efficient backpropagation through fixed-point iterations. As a demonstration, we compute the specific heat of the Ising model directly by taking the second-order derivative of the free energy obtained in the tensor renormalization group calculation. Next, we perform gradient-based variational optimization of infinite projected entangled pair states for the quantum antiferromagnetic Heisenberg model and obtain state-of-the-art variational energy and magnetization with moderate efforts. Differentiable programming removes laborious human efforts in deriving and implementing analytical gradients for tensor network programs, which opens the door to more innovations in tensor network algorithms and applications.
- Received 2 April 2019
DOI:https://doi.org/10.1103/PhysRevX.9.031041
Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.
Published by the American Physical Society
Physics Subject Headings (PhySH)
Popular Summary
Tensor networks are a powerful theoretical and computational tool for analyzing statistical and quantum physics as well as machine learning. However, optimization of tensor network states has been a long-standing problem that prevents thorough exploitation of their expressive power for quantum many-body systems. Here, we remove this roadblock by bringing the “automatic differentiation approach” used in deep learning to the field of tensor network optimization.
With the help of these deep-learning tools, one can take the derivative of energy with respect to the parameters in the tensor network, which is usually quite difficult to do. This automatic differentiation allows for gradient-based optimization of the network parameters and obtains tensor network representation of quantum many-body ground states more accurately and efficiently, without laborious human effort. To demonstrate the power of this approach, we report state-of-the-art variational results for the prototypical problem in quantum spin systems, known as the antiferromagnetic Heisenberg model on the infinite square lattice.
Breaking the optimization bottleneck opens the door to broader applications of tensor network states and more innovative algorithm designs. In the future, more of these novel techniques and ideas from deep learning or other fields may prove to be useful in computational physics and significantly advance our understanding of nature.