Absence of Barren Plateaus in Quantum Convolutional Neural Networks

Quantum neural networks (QNNs) have generated excitement around the possibility of efficiently analyzing quantum data. But this excitement has been tempered by the existence of exponentially vanishing gradients, known as barren plateau landscapes, for many QNN architectures. Recently, quantum convolutional neural networks (QCNNs) have been proposed, involving a sequence of convolutional and pooling layers that reduce the number of qubits while preserving information about relevant data features. In this work, we rigorously analyze the gradient scaling for the parameters in the QCNN architecture. We find that the variance of the gradient vanishes no faster than polynomially, implying that QCNNs do not exhibit barren plateaus. This result provides an analytical guarantee for the trainability of randomly initialized QCNNs, which highlights QCNNs as being trainable under random initialization unlike many other QNN architectures. To derive our results, we introduce a novel graph-based method to analyze expectation values over Haar-distributed unitaries, which will likely be useful in other contexts. Finally, we perform numerical simulations to verify our analytical results.

Popular Summary

With the advent of quantum computers, many different architectures have been proposed that could provide an advantage over their classical counterparts. Quantum neural networks (QNNs) are among the most promising architectures, with applications including physical simulations, optimization, and more general machine-learning tasks. Despite their tremendous potential, QNNs have been shown to exhibit a “barren plateau,” where the gradient of a cost function vanishes exponentially with system size, rendering the architecture untrainable for large problem sizes. Here, we show that one particular QNN architecture does not have this plateau.

We analyze an architecture called the quantum convolutional neural network (QCNN), which has been recently proposed to tackle classification problems on quantum data. As an example, a QCNN could be trained to classify quantum states according to the phase of matter that they belong to. We rigorously prove that QCNNs do not suffer from barren plateaus, thus highlighting them as a potential candidate architecture for achieving quantum advantage in the near term.

As the field of QNNs booms, we believe it will be important to carry out similar analyses for other candidate architectures, and the techniques developed in our work can be used as blueprints for such analyses.

Figure 1

Schematic representation of a QCNN. The input of the QCNN is an $n$-qubit quantum state ${\rho }_{\mathrm{in}}$. The state ${\rho }_{\mathrm{in}}$ is then sent through a sequence of $L$ convolutional ($C$) and pooling ($P$) layers. The convolutional layers are composed of two rows of two-qubit unitaries ($W$) acting on alternating pairs of qubits. The pooling layer is composed of pooling operators (indicated in the dashed box). In each pooling module, a qubit is measured with the measurement outcome controlling a unitary applied to a neighboring qubit ($I$). After the final pooling layer, one applies a fully connected unitary ($F$) to the remaining qubits and obtains an output state ${\rho }_{\text{out}}$ whose dimension is much smaller than that of ${\rho }_{\mathrm{in}}$. Finally, one measures the expectation value of some operator $O$ over the state ${\rho }_{\text{out}}$.

Figure 2

Tensor network representation of the QCNN. Here each unitary $W_{ij}$ in the convolutional and fully connected layers forms a 2-design. The operators $I_{ij}$ of Eq. (7) in the pooling layer do not modify the variance $\text{Var}[{\partial }_{\mu }C]$ as their action can be absorbed by the action of the unitaries in the convolutional layers. Each tensor (circle) corresponds to a $W_{ij}$, with $L=0$ corresponding to the unitary in the fully connected layer. Circle with three legs indicates that a qubit is traced out after their action.

Figure 3

GRIM modules for the QCNN architecture. (a) Basic modules $M$ from left to right; we refer to these modules as the center $M_{\mathcal{C}}$, edge $M_{\mathcal{E}}$, and middle $M_{\mathcal{M}}$ module. Note that while both $M_{\mathcal{C}}$ and $M_{\mathcal{E}}$ contain three unitaries, their contractions are different. (b) Example of the first step of the GRIM. The forward light cone ${\mathcal{L}}_F$ of $W$ can be covered with the three basic modules. Here, $W$ is indicated with a larger colored circle.

Figure 4

GRIM for the case when $W$ is in the edge of $\ell$th layer of the QCNN. (a) The forward light cone can be covered with ($\ell -1$) edge modules $M_{\mathcal{E}}$ and a single center module $M_{\mathcal{C}}$. Here, $W$ is indicated with a larger colored circle. (b) Graph obtained from the GRIM by integrating all modules $M_{\mathcal{E}}$ in $\mathrm{ar}{\mathcal{L}}_F$ for the case in (a). Graph ${\mathcal{G}}_w$ obtained via GRIM. Here the edge coefficients ${\lambda }_{i,j}$ are real and positive numbers such that ${\lambda }_{i,j}\in (0,1)$. Moreover, for this case the number of nodes is ($\ell +1$), and the structure of the graph is such that one can easily obtain the graph for any value of $\ell$.

Figure 5

GRIM for the case when $W$ is in the center of $\ell$th layer of the QCNN. (a) The unitaries in ${\mathcal{L}}_F$ can be covered by $\ell$ center modules $M_{\mathcal{C}}$. (b) Schematic representation of the tensor contraction $N_1$ of Eq. (19). Here we also show the recursive relation between the operators ${\tilde{O}}_{\ell }$. (c) Tensor network representation of Eq. (25) obtained by integrating the $({\ell }^{\prime }+1)$th center module. The operators $p_2$ and $p_3$ correspond to contractions over different qubits, as indicated by the bold lines. (d) Graph ${\mathcal{G}}_w$ obtained from the GRIM for the case in (a).

Figure 6

Schematic circuits of a pooling-based QCNN. The left module belongs to the layer that acts prior to one which contains the right module. As shown, after each pooling layer, half the qubits are measured. In this special case, the unitaries in the convolutional and fully connected layers of the QCNN act trivially.

Figure 7

Ansatz and results for the numerical simulations of the QCNN. (a) Ansatz for the two-qubit unitary $W_{ij}$. Here $R_G=e^{-iZ{\theta }_1/2}{e}^{-iY{\theta }_2/2}{e}^{-iZ{\theta }_3/2}$ is a general single-qubit rotation parametrized by three angles, and $X$, $Y$ are Pauli operators. In addition, $R_y$ and $R_z$ denote single-qubit rotations about the $y$ and $z$ axes, respectively. (b) Heuristical scaling of the variance of the cost function partial derivative $\text{Var}[{\partial }_{\mu }C]$. The $y$ axis is in a log scale. The solid (dashed) curve represents results for correlated (uncorrelated) unitaries in the convolutional layers. Error bars represent the standard deviation over 16 repetitions. Here we can see that $\text{Var}[{\partial }_{\mu }C]$ is always larger when the unitaries are correlated. As $n$ increases, we see that the curves are sublinear, indicating a subexponential scaling for $\text{Var}[{\partial }_{\mu }C]$.

Figure 8

Examples of the first step of the GRIM. Here we can cover all of the forward light cone ${\mathcal{L}}_F$ using three basic modules repeated $\ell$ times. Panels (a) and (b), respectively, correspond to the cases when the unitary $W$ is in first sublayer and second sublayer of the $\ell$th QCNN layer. In all cases, $W$ is indicated by a larger orange circle, and the forward light cone is shaded in yellow. For the panels in (a), $W$ is not included in any module, while in (b) $W$ belongs to the first module $M$.

Figure 9

Tensor network representation of ${\mathrm{\Omega }}_{\mathit{q}\mathit{p}}$ and $\mathrm{\Delta }{\mathrm{\Omega }}_{\mathit{q}\mathit{p}}^{{\mathit{q}}^{\prime }{\mathit{p}}^{\prime }}$. Here we consider the case where $W$ is in the middle of the $\ell$th layer.

Figure 10

(a) Schematic representation of the $p_{\alpha ,\beta ,s}$ coefficients. The contractions of the operators $|\mathit{p}⟩⟨\mathit{q}|$ lead to the terms $p_{\alpha ,\beta ,s}$ according to Eq. (b13). (b) Representative graph obtained after integrating the modules in the forward light cone ${\mathcal{L}}_F$. The node $N_{\alpha }$ is connected to the node $N_{\beta }$ via an (oriented) arrow. Each edge is associated with an operator $e_{\alpha ,\beta ,s}=a_{\alpha ,\beta ,s}{p}_{\alpha ,\beta ,s}$.

Figure 11

(a) Schematic representation of the backward light cone ${\mathcal{L}}_B$ (shaded in yellow) and the effective light cone ${\tilde{\mathcal{L}}}_B$ (shaded in a darker yellow). Here, the width of ${\mathcal{L}}_B$ increases with the number of layers. To avoid integrating all the unitaries in ${\mathcal{L}}_B$, we define an effective light cone ${\tilde{\mathcal{L}}}_B$ which can be covered by modules according to the GRIM. (b) Tensor network representation of the unitary ${\tilde{\sigma }}_w$. The large red rectangular tensor corresponds to $\sigma$. Here we can see that the unitaries in $V_L$ which are not in ${\mathcal{L}}_B$ simplify to identity when computing ${\mathrm{Tr}}_{\overline{w}}[V_L\sigma V_L^{†}]$. We mark such identities with an unshaded circle. (c) Integrating the unitaries in $V_{{\tilde{\mathcal{L}}}_B}^{(L-1)}$ leads to a summation of terms according to Eq. (b35). Here, one of these terms corresponds to ${ϵ}_{{\tilde{\sigma }}_w^{(\ell +2)}}$. As schematically shown, in the computation of ${ϵ}_{{\tilde{\sigma }}_w^{(\ell +2)}}$ there are many gates that simplify to identity. These gates are indicated here as shallow circles.

Figure 12

Effective backward light cone ${\tilde{\mathcal{L}}}_B$ for the case when $W$ is in the first sublayer or the $\ell$th layer of the QCNN. Here, ${\tilde{\mathcal{L}}}_B$ can be covered by $(L-\ell )$ modules $M_{\mathcal{M}}$.

Figure 13

(a) The unitary $W$ is in the edge of the second sublayer. Here all unitaries in the backward light cone ${\tilde{\mathcal{L}}}_B$ can be grouped in modules $M_{\mathcal{M}}$ except for the initial unitary $W_{\mathrm{init}}$. (b) The unitary $W$ is in the second sublayer but not in its edge. The backward light-cone structure is similar to the one in panel (a).

Figure 14

By integrating a single unitary in each sublayer, one can obtain a reduced state $\widehat{\sigma }$ in the subspace of any odd pair of qubits in ${\mathcal{L}}_B$.

Figure 15

Graphs ${\mathcal{G}}_w$ obtained through the GRIM by integrating (a) $\ell$ center modules $M_{\mathcal{C}}$, (b) $\ell$ edge modules $M_{\mathcal{E}}$, (c) $\ell$ middle modules $M_{\mathcal{M}}$. In panels (b) and (c), we indicate how the wires are labeled. (d) Transition between the middle and the edge graphs of (b) and (c). All the nodes in the middle graph are connected to the nodes $N_2$ and $N_4$ on the edge graph. The node $N_2$ of the middle graph is also connected to a node ${\widehat{N}}_1$ that connects to the rest of the edge graph.

Figure 16

Quantum circuit to implement an arbitrary unitary on two qubits. As shown in Ref. [63], $W_{ij}$ is universal (in that it can implement any two-qubit gate), and optimal (in that it contains the minimal number of cnot gates and parameters, i.e., three cnot gates and 15 parameters). Here, $R_G=e^{-iZ{\theta }_1/2}{e}^{-iY{\theta }_2/2}{e}^{-iZ{\theta }_3/2}$ is a general single-qubit rotation, and $R_{\mu }$ with $\mu \in \{y,z\}$ is a single-qubit rotation of the Bloch sphere about axis $\mu$.