Measurement-Based Feedback Quantum Control with Deep Reinforcement Learning for a Double-Well Nonlinear Potential

Closed loop quantum control uses measurement to control the dynamics of a quantum system to achieve either a desired target state or target dynamics. In the case when the quantum Hamiltonian is quadratic in $x$ and $p$, there are known optimal control techniques to drive the dynamics toward particular states, e.g., the ground state. However, for nonlinear Hamiltonian such control techniques often fail. We apply deep reinforcement learning (DRL), where an artificial neural agent explores and learns to control the quantum evolution of a highly nonlinear system (double well), driving the system toward the ground state with high fidelity. We consider a DRL strategy which is particularly motivated by experiment where the quantum system is continuously but weakly measured. This measurement is then fed back to the neural agent and used for training. We show that the DRL can effectively learn counterintuitive strategies to cool the system to a nearly pure “cat” state, which has a high overlap fidelity with the true ground state.

Figure 1

(a) The working of a DRL actor-critic model. The agent consists of two networks, actor and critic, where the actor decides the action to be applied on the environment based on the suggestion made by the critic network that computes the value function based on the reward and state obtained from the DRL environment, that includes the physics of the problem—the quantum dynamics and state of particle moving in a DW and modulation function to alter quantum dynamics and the reward estimation function based on the observables (measurement results). (b) the Wigner function distribution for the ground state of the DW, and the corresponding probability distribution on position (c) and Fock-number basis (d). The ground state of the DW is an even parity state.

Figure 2

(a) Effectiveness of the agent’s learning process as a function of measurement strength $\mathrm{\Gamma }$, indicating the existence of a “sweet spot” around $\mathrm{\Gamma }\sim 0.1$. We plot the maximum (shown in blue lines) and mean (shown in brown lines) episodic $\tilde{F}$ of ten successive deterministic episodes where $\tilde{F}$ represents the mean fidelity over an episode using trained agents, (b) the episodic time evolution of the mean reward $\tilde{R}(t)$ during the training of the agent when $\mathrm{\Gamma }=0.1$ [light (dark) blue includes (average) noise], (c) episodic mean fidelity $\tilde{F}$ of the instantaneous $\rho (t)$ with the ground state of the DW, (d) measurement current $I(t)$ illustrating that a trained agent is able to keep the wave function near the well minima (the conditional average $⟨x_c^2⟩$ is shown in brown), and (e) the corresponding variation of fidelity for a trained episode, for a deterministic episode of the trained agent. Similar performance can be obtained using fidelity as the reward function (see Supplemental Material [51]).