Analytic Theory for the Dynamics of Wide Quantum Neural Networks

Parametrized quantum circuits can be used as quantum neural networks and have the potential to outperform their classical counterparts when trained for addressing learning problems. To date, much of the results on their performance on practical problems are heuristic in nature. In particular, the convergence rate for the training of quantum neural networks is not fully understood. Here, we analyze the dynamics of gradient descent for the training error of a class of variational quantum machine learning models. We define wide quantum neural networks as parametrized quantum circuits in the limit of a large number of qubits and variational parameters. Then, we find a simple analytic formula that captures the average behavior of their loss function and discuss the consequences of our findings. For example, for random quantum circuits, we predict and characterize an exponential decay of the residual training error as a function of the parameters of the system. Finally, we validate our analytic results with numerical experiments.

Figure 1

Concentration of the QNTK on four qubits as a function of the circuit depth $L$. We pick different values of $L$ (up to 40) in the randomized Ansatz, defined in Eq. (20). We sample over 1000 variational angles from Eq. (20) independently and uniformly over $[0,2\pi ]$. We plot $\mathrm{\Delta }K/\overline{K}$ versus $L$, which verifies the analytic scaling of $L^{-1/2}$, as derived in Eq. (15). In the inset, we plot $\overline{K}$ versus $L$, where the dashed line represents the theoretical values of $\overline{K}$ from Eq. (9), green dots represent the numerical values of $\overline{K}$, and the blue error bars represent $\mathrm{\Delta }K$.

Figure 2

Residual training error $ϵ$ versus the gradient descent steps $t$ for two qubits and $L=64$ for the random Ansatz defined in Eq. (20). We use random initial angles in $[0,2\pi ]$ and perform the gradient descent experiment with a learning rate $\eta ={10}^{-4}$ with 1000 steps. For 50 different initializations, we plot the dynamics of $ϵ(t)$, the theoretical prediction for the average dynamics of $ϵ(t)$, and the numerical values for the averaged $ϵ(t)$.

Figure 3

Lowest eigenvalue for the kernel of a supervised learning task defined on four qubits and $L=64$, as a function of the training data size $|\mathcal{A}|$. The predicted linear relation $[2L(D-|\mathcal{A}|)/{(D^2-1)}^2][D\mathrm{Tr}(O^2)-{\mathrm{Tr}}^2(O)]$ (orange solid line) agrees with numerical estimations with 50 independent instances of the Ansatz.