Expressivity of quantum neural networks

In this work, we address the question whether a sufficiently deep quantum neural network can approximate a target function as accurate as possible. We start with typical physical situations that the target functions are physical observables, and then we extend our discussion to situations that the learning targets are not directly physical observables, but can be expressed as physical observables in an enlarged Hilbert space with multiple replicas, such as the Loschmidt echo and the Rényi entropy. The main finding is that an accurate approximation is possible only when all the input wave functions in the dataset do not span the entire Hilbert space that the quantum circuit acts on, and more precisely, the Hilbert space dimension of the former has to be less than half of the Hilbert space dimension of the latter. In some cases, this requirement can be satisfied automatically because of the intrinsic properties of the dataset, for instance, when the input wave function has to be symmetric between different replicas. And if this requirement cannot be satisfied by the dataset, we show that the expressivity capabilities can be restored by adding one ancillary qubit at which the wave function is always fixed at input. Our studies point toward establishing a quantum neural network analogy of the universal approximation theorem that lays the foundation for expressivity of classical neural networks.

Figure 1

(a) Conventional QNN structure. (b) QNN with one ancillary qubit. (c) QNN with two duplicated replicas of inputs. (d) QNN with two duplicated replicas of input and one ancillary qubit. Here $|\psi \rangle$ are input wave function in the dataset, $\widehat{\mathcal{U}}$ denotes the unitary rotation by quantum circuit, and the detector denotes the readout qubit. The red qubit with label $a$ denotes the ancillary qubit at which the wave function is always fixed.

Figure 2

The loss function as a function of the training epoch. The solid lines and the dashed lines are the loss on the training dataset and the validation dataset, respectively. The target functions are, respectively, the magnetization (a), the Loschmidt echo (b), the second Rényi entropy (c) and the third Rényi entropy (d). For the input wave function $|{\psi }^l\rangle$ in the dataset, the number of qubit $n=3$ for (a), and $n=2$ for (b), (c) and (d). [35]. For the Rényi entropy discussed in (c) and (d), one qubit is taken as $\mathcal{A}$ and the other qubits are taken as $\mathcal{B}$. In the legends, $\mathrm{S}$, $\mathrm{D}$ and $\mathrm{T}$, respectively, denote single, double, and triple replicas as input for quantum circuit, and $+\mathrm{A}$ means that one ancillary qubit is added. Here we use the brick wall architecture made of two-qubit gate for quantum circuit.