Information Scrambling in Quantum Neural Networks

The quantum neural network is one of the promising applications for near-term noisy intermediate-scale quantum computers. A quantum neural network distills the information from the input wave function into the output qubits. In this Letter, we show that this process can also be viewed from the opposite direction: the quantum information in the output qubits is scrambled into the input. This observation motivates us to use the tripartite information—a quantity recently developed to characterize information scrambling—to diagnose the training dynamics of quantum neural networks. We empirically find strong correlation between the dynamical behavior of the tripartite information and the loss function in the training process, from which we identify that the training process has two stages for randomly initialized networks. In the early stage, the network performance improves rapidly and the tripartite information increases linearly with a universal slope, meaning that the neural network becomes less scrambled than the random unitary. In the latter stage, the network performance improves slowly while the tripartite information decreases. We present evidences that the network constructs local correlations in the early stage and learns large-scale structures in the latter stage. We believe this two-stage training dynamics is universal and is applicable to a wide range of problems. Our work builds bridges between two research subjects of quantum neural networks and information scrambling, which opens up a new perspective to understand quantum neural networks.

Figure 1

(a) Schematic of a quantum circuit with brick-wall geometry. Here, the network has $n=5$ qubits and depth $l=4$. All two-qubit gates form a giant unitary transformation $\widehat{U}$ that distills the information from the input qubits and encodes it into one output qubit. The inverse process is that the information of one output qubit is scrambled into input qubits by ${\widehat{U}}^{†}$. $A$ is the output subsystem, and $C$ and $D$ are input subsystems in the definition of the tripartite information. (b) Illustration for the operator-state mapping in the definition of tripartite information. Each leg may represent multiple qubits.

Figure 2

(a) Training loss $\mathcal{L}$ and tripartite information $I_3(A,C,D)$ as functions of the training epoch. The shaded area represents one standard deviation. (b) Finite difference of training loss $\mathrm{\Delta }\mathcal{L}$ and tripartite information $\mathrm{\Delta }{I}_3$ as functions of the training epoch. The dotted vertical line indicates the boundary between two training stages, which is determined as the maximum of the averaged $I_3$ given by $\mathrm{\Delta }{I}_3=0$. All results are averaged over 20 different random initializations. The network has $n=9$ qubits and depth $l=6$. The training and validation dataset contains $N=2500$ and 500 wave function-magnetization pairs, respectively, sampled from random Hamiltonian ensemble, where random parameters are distributed uniformly within $J_{ij}/J\in [-1,0]$, $K_{ij}/J\in [-1,1]$, $g_i/J\in [-6,6]$, and $h/J\in [-0.04,0.04]$. $J$ is the energy unit. The learning rate is $\lambda ={10}^{-2}$. Here and in the rest of the Letter, the input subsystem size $|C|=5$.

Figure 3

(a) Two-point correlation function $C_2(i)$ as a function of the training epoch and the input qubit $i$ for a typical initialization. (b) Tripartite information $I_3(A,C,D)$ as a function of the training epoch for different initializations and learning rates. All solid lines are trained under learning rate $\lambda ={10}^{-2}$. The transparent orange lines are trained with the same initialization as the solid orange line but with learning rates $\lambda =6,8,12$, and $14\times {10}^{-3}$. The average slope for the four initializations shown here is plotted in the inset as a function of the learning rates. The error bars represent the standard deviations of fitted slopes for the fixed learning rate but different initializations.

Figure 4

(a) Training loss and tripartite information as functions of the training epoch for a typical initialization. (b) Loss functions on the artificial test dataset with “ferromagnetic domain” of size $\mathcal{D}=3$ and 5 for the same training instance as Fig. 4.