Benchmarking characterization methods for noisy quantum circuits

Effective methods for characterizing the noise in quantum computing devices are essential for programming and debugging circuit performance. Existing approaches vary in the information obtained as well as the amount of quantum and classical resources required, with more information generally requiring more resources. Here we benchmark the characterization methods of gate set tomography, Pauli channel noise reconstruction, and empirical direct characterization for developing models that describe noisy quantum circuit performance on a 27-qubit superconducting transmon device. We evaluate these models by comparing the accuracy of noisy circuit simulations with the corresponding experimental observations. We find that the agreement of noise model to experiment does not correlate with the information gained by characterization and that the underlying circuit strongly influences the best choice of characterization approach. Empirical direct characterization scales best of the methods we tested and produced the most accurate characterizations across our benchmarks.

Figure 1

Graphical representation of randomly compiling a quantum circuit [16, 18]. Colored boxes represent easy gates; grey multiqubit gates are considered to be hard gates in this example. (a) Starting with a quantum circuit, (b) we inject twirling gates, which are depicted by blue squares with dashed lines. (c) Then these gates are compiled together to form a randomly compiled circuit. This process is repeated for a set of randomly selected twirling gates to generate a set of $n$ twirled quantum circuits which together represent the randomly compiled quantum circuit.

Figure 2

An illustration of the experiments we run, the models we extract from those experimental results, and the simulations we run using these models. Bare circuits (BCs) are circuits that are not randomly compiled (RC).

Figure 3

A graphical representation of the register layout of the 27-qubit toronto QPU at the time of data collection. Each node corresponds to a register element and directional edges indicate the availability of a programmable two-qubit cross-resonance gate.

Figure 4

Error rate in readout of state 0 on toronto using the EDC methodology.

Figure 5

Error rate in readout of state 1 on toronto using the EDC methodology.

Figure 6

Depolarizing error rate for the $X$ gate on toronto using the EDC methodology.

Figure 7

Rate expected outcomes were observed from BC GHZ-state preparation circuits executed on toronto across a 12-hour period. The decay function of the best performing set (1555) is $1.112e^{-0.0834x}$ with $R^2=0.989$ for register size $x$ [26].

Figure 8

Rate expected outcomes were observed from RC GHZ-state preparation circuits executed on toronto across a 12-hour period. Decay function of the best performing set (first set) is $1.148e^{-0.12x}$ with $R^2=0.9855$ for register size $x$ [26].

Figure 9

TVD between experiment results and noiseless GHZ-state preparations. RC circuits are limited to the first 20 qubits. The RC TVD increases with register size $x$ as $0.2418e^{0.0801x}$ with $R^2=0.978$ and the BC TVD increases as $0.2625e^{0.053x}$ with $R^2=0.963$ [26].

Figure 10

TVD between experiment results and noiseless GHZ results. For each size of GHZ-state preparation, the results are trimmed to the first two qubits. The RC TVD increases with register size $x$ as $0.0906e^{0.0445x}$ with $R^2=0.999$ and the BC TVD increases as $0.0889e^{0.024x}$ with $R^2=0.999$ [26].

Figure 11

The Pauli transfer matrix estimated for cnot on qubits 0 and 1 from GST [22]. The color scale ranges from red for values close to 1 to blue for values close to $-1$.

Figure 12

Readout matrix representing results from GST SPAM estimates.

Figure 13

Total error rates for each cnot characterized by KNR. Values represent the sum of all error types measured by the protocol.

Figure 14

Estimate of the depolarizing component of the noise channels estimated by KNR.

Figure 15

Readout error rates for toronto estimated by EDC.

Figure 16

Error rates of cnot for toronto estimated by EDC. We observe spatial variability across the register and asymmetry in control versus target qubits in each coupling.

Figure 17

TVD to experiment of composite noise models constructed from error rates estimated using KNR. Error bars are calculated as the standard deviation across 100 trials of the Bell-state preparation circuit distributions. The full model with readout error based on just two circuits provided results closest to experiment as measured by TVD.

Figure 18

TVD to experiment of composite noise models constructed from error rates estimated using EDC. Error bars are calculated as the standard deviation across 100 trials of the Bell-state preparation circuit distributions. Although the readout-only model provided results closest to experiment as measured by TVD, it is within error bars of the full model which is likely because the estimates for depolarizing noise in the gates were relatively low.

Figure 19

TVD between experiment and noisy simulation of the Bell state on qubits 0 and 1 using noise models constructed from GST, KNR, and EDC protocols. Error bars represent the standard deviation across 100 trials of Bell-state simulation distributions. The error for the noiseless and self-simulation cases is calculated as the error propagation in TVD from the distributions.

Figure 20

Total variation distance between experiment results and simulation results. Solid lines indicate TVDs calculated with randomly compiled results; dashed lines indicate TVDs with bare circuits. TVD for the noiseless case is calculated between experiment and results exactly split between an all-zero state and an all-one state. TVD for the “self-simulation” case is calculated between experiment and GHZ results executed on toronto during the same 12-hour period.

Figure 21

Bernstein-Vazirani results from experiment and noisy simulation. The accuracy of the secret string is the number of times the correct encoding is measured out of all measurements of the BV circuit. Higher accuracy indicates less noise present in the quantum circuit.

Figure 22

Time taken for noisy GHZ simulation as a function of qubit count. Noisy BC GHZ circuit simulations were switched from the local laptop to the IBM Q simulator backend for $19+$ qubits due to the computational intensity of the largest GHZ circuit simulations. The IBM Q simulation backend has a limit of 32 qubits [19].

Figure 23

Bell-state preparation circuit.

Figure 24

Implementation of the Bernstein-Vazirani algorithm shown for an encoded binary secret string of 101.