Machine-learning engineering of quantum currents

The design, accurate preparation, and manipulation of quantum states in quantum circuits are essential operational tasks at the heart of quantum technologies. Nowadays, circuits can be designed with physical parameters that can be controlled with unprecedented accuracy and flexibility. However, the generation of well-controlled current states is still a nagging bottleneck, especially when different circuit elements are integrated together. In this work, we show how machine learning can effectively address this challenge and outperform the current existing methods. To this end, we exploit deep reinforcement learning to prepare prescribed quantum current states in circuits composed of lumped elements. To highlight our method, we show how to engineer bosonic persistent currents as they are relevant in different quantum technologies as cold atoms and superconducting circuits. We demonstrate the use of deep reinforcement learning to rediscover established protocols, as well as solving configurations that are difficult to treat with other methods. With our approach, quantum current states characterized by a single winding number or entangled currents of multiple winding numbers can be prepared in a robust manner, superseding the existing protocols.

Figure 1

(a), (b) Deep reinforcement learning to optimize generation of quantum states of currents. The quantum system is a ring-shaped circuit comprising $L$ lumped parameters schematized as lattice sites. The wave function resides at site $m$ with a local potential $P_m\left(t\right)$ that can be changed in time. The full control protocol (FCP) adjusts the potential at all sites individually in discrete time steps $t_n$ to generate a current efficiently. To optimize the protocol, the neural network takes the potentials $P(t<t_n)$ at earlier time steps and returns the potential $P_m\left(t_n\right)$ to be applied to the quantum system in the next time step. A measure $\left.〈\mathrm{\Psi }\right|\widehat{O}\left|\mathrm{\Psi }\right.〉$ given by observable $\widehat{O}$ is measured and used to train the neural network. This process is repeated until convergence. For further details, see Appendix pp6. (c) Example driving potential found by the optimization algorithm.

Figure 2

(a) Generating currents by driving the potential of a ring circuit. We plot the time evolution of the fidelity $F={\left|\langle \mathrm{\Psi }\left(T\right)|{\mathrm{\Psi }}_{\text{target}}\rangle \right|}^2$ obtained with two different protocols, comparing different numbers of particles [Bose-Hubbard model Eq. (1) with $L=12$ sites, $U=J$, target state $|{\mathrm{\Psi }}_{\text{target}}\rangle ={\left|k=1\right.〉}^{\otimes N_{\text{p}}}$]. The first protocol (barrier) stirs the wave function by moving a single-site barrier ($P_{\text{B}}=2J$ for $N_{\text{P}}=1,2$, $P_{\text{B}}=1.6J$ for $N_{\text{P}}=3$) at a constant speed $v=0.53J$ (see the curves on the right of the vertical dotted line), with the Bose-Hubbard model ($L=12$ ring sites, interaction $U=J$). The second protocol [full control protocol (FCP)] is fully controlling the potential at every lattice site in time (see the curves on the left of the vertical dotted line) with $N_{\text{T}}=6$ control steps: solid blue curves, $N_{\text{p}}=1$; dashed red line, $N_{\text{p}}=2$; dotted green curves, $N_{\text{p}}=3$. Limit of large particle number with quantum phase model (QPM) Eq. (2) with $L=7$, $U=3J_{\text{E}}$, and target state $|{\mathrm{\Psi }}_{\text{target}}\rangle =|{\mathrm{\Psi }}_{\text{QP}}\left({\mathrm{\Phi }}_1\right)\rangle$ (see Appendix pp1): long-dashed black curve. (b) Minimal time $T_{\text{min}}$ required to create rotational states above a threshold fidelity ($F_{\text{min}}=0.95$ for $N_{\text{p}}=1$; for more elusive higher particle number states, $F_{\text{min}}=0.85$) for different values of barrier amplitude $P_{\text{B}}$. (c) Maximal fidelity achieved when rotating the barrier with amplitude $P_{\text{B}}$ and speed $v$ for $N_{\text{p}}=1$ particles. We find that the best rotation speed of the barrier is at $v\approx 0.5J$.

Figure 3

Current generation by moving a single $\delta$-like potential (with constant amplitude $P=J$). We compare two protocols: Moving potential at constant speed $v=0.52J$ (solid line) or optimize with deep RL (dashed line). Over $n_t=100$ time steps, the RL agent can either move the barrier by one site, or keep it at the current position, trained over 50 000 epochs. (a) Fidelity of the best found protocol to create the phase winding state $\mathrm{\Omega }=1$ for $N_p=1$. (b) Barrier position in time.

Figure 4

(a)–(c) Full control with continuous control: Potential at every site can take arbitrary values between $-J<P_j<J$. Generation of entangled superpositions of current states of type $\left|\text{EC}\right.〉=\frac{1}{\sqrt{N_{\text{C}}}}{\sum }_{k\in \mathrm{\Omega }}{\left|k\right.〉}^{\otimes N_{\text{p}}}$ of a set of winding numbers $\mathrm{\Omega }=\{k_1,k_2,\cdots k_{N_{\text{C}}}\}$ using deep RL. (a) Fidelity as a function of protocol time $T$ for $N_{\text{p}}=2$ particles, $N_{\text{T}}=6$ time steps, and $U=J$. (b) Fidelity for varying time steps $N_{\text{T}}$ ($N_{\text{p}}=2$, $U=J$, $\mathrm{\Omega }=\{0,1\}$). (c) Fidelity for the limit of large particle number (QPM) with FCP protocol for $N_{\text{T}}=6$, $L=7$, $\left|P_m\left(t\right)\right|\le J_E$ and limiting number fluctuations at $\mathrm{\Delta }{\widehat{Q}}_m\le 2$ and target state $|{\mathrm{\Psi }}_{\text{target}}\rangle =|{\mathrm{\Psi }}_{\text{QP}}\left({\mathrm{\Phi }}_1\right)\rangle$. The ground state of the QPM has a broad winding number distribution and thus there is a finite initial fidelity at $T=0$. (d) Full control protocol with discretized amplitudes: Local potential can take either the value $P_j\left(t\right)=0$ or $P_j\left(t\right)=J$. Fidelity to create phase winding state $\mathrm{\Omega }=1$ for $N_p=1$, for varying protocol time $T$ and number time steps $N_{\text{T}}$.

Figure 5

Comparison of fidelity with entangled state $F={\left|\langle \mathrm{\Psi }|\text{ES}\rangle \right|}^2$ (for figure index 1) and certification measure $W_{\mathrm{\Psi }}$ (for index 2) for $L=12$ sites for various parameters. Same parameters as in Fig. 4. We optimize for equal weight entangled states of $N_{\text{C}}$ winding number $k$ of type $\left|\text{ES}\right.〉=\frac{1}{\sqrt{N_{\text{C}}}}{\sum }_{k\in \mathrm{\Omega }}{\left|k\right.〉}^{\otimes N_{\text{p}}}$ of a set of winding number $\mathrm{\Omega }=\{k_1,k_2,\cdots ,k_{N_{\text{C}}}\}$. (a) Varying time $T$ to generate different entangled states for $N_{\text{p}}=2$ particles, $N_{\text{T}}=6$ time steps, and $U=J$. (b) For varying time steps $N_{\text{T}}$ to reach state $\mathrm{\Omega }=\{0,1\}$ for $N_{\text{p}}=2$ particles. (c) Interaction $U$ dependence for different types of states for $N_{\text{T}}=6$ time steps and protocol time $T=9/J$. (d) Total protocol time $T$ for $N_{\text{T}}=4$ time steps to generate entangled superposition state of winding number $\mathrm{\Omega }=\{0,1\}$.

Figure 6

Generation of entangled superpositions of current states of type $\left|\text{EC}\right.〉=\frac{1}{\sqrt{N_{\text{C}}}}{\sum }_{k\in \mathrm{\Omega }}{\left|k\right.〉}^{\otimes N_{\text{p}}}$ of a set of winding numbers $\mathrm{\Omega }=\{k_1,k_2,\cdots ,k_{N_{\text{C}}}\}$ using GRAPE. (a) Fidelity as a function of protocol time $T$ for $N_{\text{p}}=2$ particles, $N_{\text{T}}=6$ time steps, and $U=J$. (b) Fidelity for varying time steps $N_{\text{T}}$ ($N_{\text{p}}=2$, $U=J$, $\mathrm{\Omega }=\{0,1\}$). (c) Fidelity as a function of interaction $U$ for different types of states ($N_{\text{p}}=2$, $N_{\text{T}}=6$, $T=9/J$). (d) Fidelity as a function of protocol time $T$ for three different particle numbers $N_{\text{P}}$ ($N_{\text{T}}=4$, $U=J$, $\mathrm{\Omega }=\{0,1\}$). All data for $L=12$ sites. The GRAPE algorithm is run multiple times to avoid solutions that became stuck in local extrema.

Figure 7

Full control driving; local potential can take continuous values between $P=-J$ and $P=J$, changing in discrete time steps. The protocol is optimized with GRAPE. Then, the local potential of the optimal protocol is perturbed with random noise, sampled from uniform distribution between $[-\delta P/2,\delta P/2]$, sampled for every site and time step. Each point is sampled from 100 noise realizations; dots show the mean value and error bars show the standard deviation of fidelity $F$. (a) Fidelity to create the phase winding state $\mathrm{\Omega }=1$ for $N_p=1$, for varying random perturbation $\delta P$ and protocol time $T$. Robustness against noise decreases with increasing time $T$. (b) Varying protocol time steps $N_t$ increases with the number of time steps for $\mathrm{\Omega }=1$ and $N_p=1$. (c) Varying $T$ for $N_{\text{p}}=2$, $U=J$, $N_{\text{T}}=6$ and target state $\mathrm{\Omega }=1$. (d) Same parameters for entangled superposition state of $\mathrm{\Omega }=\{0,1\}$.

Figure 8

Fidelity of generating the current state for the quantum phase model. The initial state is the ground state without flux, while the target state is the ground state of the model with one flux quantum. The plot shows the fidelity of reaching the target state $\left|{\mathrm{\Psi }}_{\text{QP}}\left({\mathrm{\Phi }}_1\right)\right.〉$ for different protocol times $T$. We restrict the local Hilbert space $\mathrm{\Delta }{\widehat{Q}}_m=4$. We choose $N_{\text{T}}=8$, $L=5$, and potential $\left|P_{\text{max}}<J_{\text{E}}\right|$.

Figure 9

(a) Full control protocol with discretized amplitudes: The local potential can take either the value $P_j\left(t\right)=0$ or $P_j\left(t\right)=J$. Fidelity to create the phase winding state $\mathrm{\Omega }=1$ for $N_p=1$, for varying protocol time $T$ and number of time steps $N_{\text{T}}$. (b) Resulting protocol for discretized driving amplitudes.

Figure 10

Generation of (entangled) superposition states of zero and one rotational quantum $\mathrm{\Omega }=\{0,1\}$ in a ring lattice with $L=12$ sites. Evolution of certification measure [Eq. (B1)] during driving. We compare two different protocols: A barrier localized at a single site moving at constant speed [right curves in (a)] or fully controlling the potential (FCP) of every lattice individually [left curves of (a)]. (b) Minimal time $T_{\text{min}}$ required to create rotational states above a threshold fidelity ($W_{\text{min}}=0.95$ for $N_{\text{p}}=1$, otherwise $W_{\text{min}}=0.8$) for different values of barrier amplitude. We find the best rotation speed of the barrier is at $v\approx 0.5J$. (c) Best protocol for rotating barrier. Curve shows the barrier position over time. (d) Best protocol for full control over lattice potentials for a protocol of two time steps $n_{\text{t}}$. Barrier and full control protocols shown calculated for $N_{\text{p}}=1$ particles.

Figure 11

Neural network to optimize protocols to generate quantum states. Deep learning relies on representing a highly complex function (e.g., the quality of the driving protocol) with a neural network, and optimizes it using observable data (e.g., measurement outcomes). The quantum system is a lattice ring with $L$ sites where particles can hop between neighboring sites with strength $j$. Each site $m$ has a local potential $P_m\left(t_n\right)$ that can be modulated in discrete time steps $t_n$. The neural network controls the evolution of the quantum system by adjusting $P_m\left(t_n\right)$ and optimizes the parameters over many runs. The neural network performs stepwise evolution of the quantum system in $N_{\text{T}}$ discrete time steps $t_{\text{n}}$ over total runtime $T$. It uses the chosen potentials of previous time steps as an input [state $s\left(t_n\right)$], and it returns the potentials to be chosen at the next step [action $a\left(t_n\right)$] by sampling them from a Gaussian distribution. The training is performed by using a measure for the quantum state [reward $r\left(t_n\right)$].

Figure 12

Optimization of the FCP protocol by the neuronal network over the number of epochs (number of protocol runs). We show exemplary data that generated the protocol shown in Fig. 2. The dots indicate fidelity achieved during a particular run, while the red line is the moving average over the results. (a) $N_{\text{p}}=1$, (b) $N_{\text{p}}=2$, (c) $N_{\text{p}}=3$, and (d) quantum phase model.

Figure 13

Statistics (minimum, maximum, and average certification measure $W_{\mathrm{\Psi }}$) over 20 repeated runs of the algorithm for different parameter sets. (a) Different particle numbers $N_{\text{p}}$ for $N_{\text{T}}=4$, $T=8/J$, $U=J$, and $\mathrm{\Omega }=\{0,1\}$. (b) Different protocol steps $N_T$ for a total protocol length of $T=10/J$, $\mathrm{\Omega }=\{-1,1\}$, $L=12$ sites for $N_{\text{p}}=3$ particles, and $U=J$. Driving with local potential $\left|P\right|<P_{\text{max}}=J$.