Theory for Equivariant Quantum Neural Networks

Quantum neural network architectures that have little to no inductive biases are known to face trainability and generalization issues. Inspired by a similar problem, recent breakthroughs in machine learning address this challenge by creating models encoding the symmetries of the learning task. This is materialized through the usage of equivariant neural networks the action of which commutes with that of the symmetry. In this work, we import these ideas to the quantum realm by presenting a comprehensive theoretical framework to design equivariant quantum neural networks (EQNNs) for essentially any relevant symmetry group. We develop multiple methods to construct equivariant layers for EQNNs and analyze their advantages and drawbacks. Our methods can find unitary or general equivariant quantum channels efficiently even when the symmetry group is exponentially large or continuous. As a special implementation, we show how standard quantum convolutional neural networks (QCNNs) can be generalized to group-equivariant QCNNs where both the convolution and pooling layers are equivariant to the symmetry group. We then numerically demonstrate the effectiveness of a $\mathbb{S}\mathbb{U}(2)$-equivariant QCNN over symmetry-agnostic QCNN on a classification task of phases of matter in the bond-alternating Heisenberg model. Our framework can be readily applied to virtually all areas of quantum machine learning. Lastly, we discuss about how symmetry-informed models such as EQNNs provide hopes to alleviate central challenges such as barren plateaus, poor local minima, and sample complexity.

Popular Summary

Most currently used quantum neural network architectures have little-to-no inductive biases, leading to trainability and generalization issues. Inspired by a similar problem, recent breakthroughs in classical machine learning address this crux by creating models encoding the symmetries of the learning task. This is materialized through the usage of equivariant neural networks whose action commutes with that of the symmetry.

In this work, we import these ideas to the quantum realm by presenting a general theoretical framework to understand, classify, design, and implement equivariant quantum neural networks. As a special implementation, we show how standard quantum convolutional neural networks (QCNN) can be generalized to group-equivariant QCNNs where both the convolutional and pooling layers are equivariant under the relevant symmetry group.

Our framework can be readily applied to virtually all areas of quantum machine learning and provides the hope to alleviate central challenges such as barren plateaus, poor local minima, and sample complexity.

Figure 1

A schematic representation of our main results. (a) In GQML, we start by identifying the symmetry group—or groups—that leave the data labels invariant. For the example shown, the data can be visualized on a three-dimensional sphere and the labels are invariant under the action of $\mathbb{S}\mathbb{O}(3)$. (b) Both in classical and quantum machine learning, it has been shown that models with equivariant layers often have an improved performance over nonequivariant architectures. The key feature of equivariance is that applying a rotation to the input data and sending it through the layer is the same as first sending the data through the layer and then rotating the output. On the other hand, feeding either a raw or a rotated data instance into a nonequivariant layer usually leads to distorted outputs that are not related by a rotation. (c) In this work, we provide a toolbox of methods for creating equivariant quantum neural networks (EQNNs) that can be readily used to construct quantum architectures with strong geometric priors.

Figure 2

The equivariant quantum neural network. (a) The design procedure. We consider a QML problem composed of a data set (that can either be quantum mechanical in nature or corresponding to classical data that have been encoded in quantum states) as well as a label symmetry group $G$. The first step is to define the input and output representations of $G$ at each layer, where these can be natural, faithful, nonfaithful, etc. From here, we will provide different techniques that allow us to construct the EQNN layers and control, e.g., the locality of their gates. (b) The EQNN architecture. The dashed lines indicate the representations of the symmetry group $G$ at specific stages in the EQNN, which may change between layers. At first, the input state ${\rho }_{\mathrm{in}}$ is acted upon by the representation $R^{\mathrm{in}}$. The $l\mathrm{th}$ layer of the EQNN, ${\mathcal{N}}_{{\mathit{\theta }}_l}^l$, must be $(G,R^l,R^{l+1})$-equivariant. In sum, the full architecture, $\varphi ={\mathcal{N}}_{{\mathit{\theta }}_L}^L\circ \cdots \circ {\mathcal{N}}_{{\mathit{\theta }}_1}^1$, is $(G,R^{\mathrm{in}},R^{\mathrm{out}})$-equivariant. The $(G,,R^{\mathrm{out}})$-equivariant measurement operator $O$ is in the commutant of the output representation $R^{\mathrm{out}}$. Note that if we only want the EQNN to produce an output state equivariantly or invariantly (e.g., in generative models), we can omit the measurements.

Figure 3

Different types of equivariant layers in a general architecture of EQNNs. A standard layer maps data between spaces of the same dimension. An embedding (pooling) layer maps the data to a higher-dimensional (smaller-dimensional) space. In a lifting layer, $\ker (R^{l-1})>\ker (R^l)$, while in a projection layer, $\ker (R^{l-1})<\ker (R^l)$.

Figure 4

An example of the null-space method. We demonstrate how to use the null-space method to determine the space of 1-to-1-qubit ($G$,$R^{\mathrm{in}}$,$R^{\mathrm{out}}$)-equivariant quantum channels, with $G={\mathbb{Z}}_2=\{e,\sigma \}$, $R^{\mathrm{in}}=\{\mathbb{1},X\}$ and $R^{\mathrm{out}}=\{\mathbb{1},Z\}$. (a) The matrix representation of both in and out adjoint representations of the symmetry group. (b) A basis for the eight-dimensional solution space, as well as two possible equivariant channels: $\varphi (\rho )=\mathrm{Tr}[\rho ]/2$ obtained from the solution in red and $\varphi (\rho )=(X\rho X+Z\rho Z)/2$ obtained by combining the two solutions in green.

Figure 5

An example of the twirling method. We demonstrate how to use the twirling method to determine the space of 1-to-1-qubit ($G$, $R^{\mathrm{in}}$, $R^{\mathrm{out}}$)-equivariant quantum channels, with $G={\mathbb{Z}}_2=\{e,\sigma \}$, $R^{\mathrm{in}}=\{\mathbb{1},X\}$ and $R^{\mathrm{out}}=\{\mathbb{1},Z\}$. (a) An explicit calculation using the twirling formula of Eq. (25). (b) The ancilla-based scheme for in-circuit twirling. (c) The classical-randomness scheme for in-circuit twirling. Both of the schemes in (b) and (c), detailed in Appendix pp4, recover the twirling in (a).

Figure 6

An example of the Choi-operator method. We demonstrate how to use the Choi-operator method to determine the space of 1-to-1-qubit ($G$,$R^{\mathrm{in}}$,$R^{\mathrm{out}}$)-equivariant quantum channels, with $G={\mathbb{Z}}_2=\{e,\sigma \}$, $R^{\mathrm{in}}=\{\mathbb{1},X\}$ and $R^{\mathrm{out}}=\{\mathbb{1},Z\}$. (a) Isotypic decomposition of the group representation. (b) We show that a specific choice for the block-diagonal components of $J^{\varphi }$ leads to the map $\varphi (\rho )=(X\rho X+Z\rho Z)/2$.

Figure 7

The procedure to parametrize and optimize equivariant quantum neural networks. (a) We provide techniques to parametrize both equivariant unitaries and general equivariant channels. (b) Once we have a parametrized EQNN, we can proceed to train it. Here, we feed the training data into the EQNN, the outputs of which are used to compute the loss function. Leveraging a classical optimizer, we find updates for the parameters in the EQNN. In the case of channels, if might be necessary to project the updated map to the feasible CPTP region. Note that we can use classical compilers to transform a linear combination of channels into an sequence of gates that we can implement on a quantum device (Appendix pp5). The procedure is repeated until convergence is achieved.

Figure 8

$\mathbb{S}\mathbb{U}(2)$-equivariant QCNN. (a) We consider the problem of building 2-to-2 standard equivariant channels and 2-to-1 equivariant pooling channels. In the figure, we present the respective input and output Hilbert spaces, as well as the input and output representation. (b) In an $\mathbb{S}\mathbb{U}(2)$-equivariant QCNN, we alternate between 2-to-2 channels acting on neighboring qubits and 2-to-1 equivariant pooling channels that reduce the feature space dimension.

Figure 9

The region of parameter space leading to CPTP channels. Using the null-space method, we can find a basis for all 2-to-1 $(\mathbb{S}\mathbb{U}(2),g^{\otimes 2},g)$-equivariant pooling maps. These can then be linearly combined to form a general parametrized equivariant map as in Eq. (35) and we find in Eq. (37) the region in parameters space leading to CPTP channels. Here, we depict said region as the volume of the hyperbole (red) below the plane (green).

Figure 10

The 1D bond-alternating $XXX$ Heisenberg model [Eq. (47)] and its phase diagram in terms of exchange coupling constants $J_{1,2}>0$.

Figure 11

The predicted phase diagrams. The four panels show the phase diagram of the 1D bond-alternating $XXX$ Heisenberg model for system sizes of $N=12$ and $N=13$ qubits, as reconstructed by a trained $\mathbb{S}\mathbb{U}(2)$ EQCNN or HEA QCNN. In particular, each panel shows the QCNN output when it is tested against a data set of 500 homogeneously distributed ground states. States the output of which is above (below) the optimal threshold $\tau$ (green dashed line) are colored in blue (red) and classified as belonging to the trivial (topological) phase. The training points are shown as black crosses. The vertical solid black line is the theoretical critical value $J_2/J_1=1$. The configurations leading to the panels are the following: (a) $\mathbb{S}\mathbb{U}(2)$ EQCNN, $N=12$, 60 trainable parameters, 12 training points; (b) HEA QCNN, $N=12$, 63 trainable parameters, 12 training points; (c) $\mathbb{S}\mathbb{U}(2)$ EQCNN, $N=13$, 66 trainable parameters, 12 training points; (d) HEA QCNN, $N=13$, 66 trainable parameters, 12 training points. Details about the training procedure are given in the main text.

Figure 12

The actual power of equivariance, showing the mean and variance of the testing accuracy reached by trained QCNNs on the bond-alternating Heisenberg model of (a) even and (b) odd sizes. The average is conducted on both the chosen sizes, $(6,8,10,12)$ for the even case and $(7,9,11,13)$ for the odd one, and on ten different randomly initialized training runs for each problem size. The results are plotted against the number of training data points, $N_T$. The blue circles refer to the EQCNN with the same number of parameters as the HEA QCNN (orange stars). The green circles describe the EQCNN with the minimum number of parameters possible, i.e., the architecture, as it is, in Fig. 8, with only two standard layers before each pooling layer. These plots demonstrate that equivariance provides significant improvements when, and only when, there is degeneracy in the ground space.

Figure 13

The reproduction of the “exact” QCNN architectures used in Ref. [23] for the classification of the ground states of the Haldane model: C, convolution; P, pooling; FC, fully connected. The blue three-qubit gates are Toffoli gates, with control qubits taken in the $X$ basis. The orange two-qubit gates apply a Pauli $Z$ onto the target qubit when an $X$ measurement of the control qubit results in an outcome of $-1$ but leaves the target qubit unchanged otherwise. The black two-qubit gates are cz gates. The green interleaved lines correspond to a swap of two qubits. The C-P structure is repeated $L$ times until the system is left with only three qubits.

Figure 14

The circuit for the cross-product channel. Here, $i,j,k\in \{X,Y,Z\}$. These classical random variables are drawn from $\{X,Y,Z\}$ uniformly and without replacement. That is, $\{i,j,k\}=\{X,Y,Z\}$ always but with the individual rotations randomly chosen. After performing this protocol, the reduced state on the ancilla will be equal, in expectation, to $\mathcal{C}\mathcal{P}(\rho )$. Note that ${ϵ}_{ijk}$ is the Levi-Civita symbol.