Abstract
The ability to prepare a physical system in a desired quantum state is central to many areas of physics such as nuclear magnetic resonance, cold atoms, and quantum computing. Yet, preparing states quickly and with high fidelity remains a formidable challenge. In this work, we implement cutting-edge reinforcement learning (RL) techniques and show that their performance is comparable to optimal control methods in the task of finding short, high-fidelity driving protocol from an initial to a target state in nonintegrable many-body quantum systems of interacting qubits. RL methods learn about the underlying physical system solely through a single scalar reward (the fidelity of the resulting state) calculated from numerical simulations of the physical system. We further show that quantum-state manipulation viewed as an optimization problem exhibits a spin-glass-like phase transition in the space of protocols as a function of the protocol duration. Our RL-aided approach helps identify variational protocols with nearly optimal fidelity, even in the glassy phase, where optimal state manipulation is exponentially hard. This study highlights the potential usefulness of RL for applications in out-of-equilibrium quantum physics.
- Received 12 January 2018
- Revised 1 August 2018
DOI:https://doi.org/10.1103/PhysRevX.8.031086
Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.
Published by the American Physical Society
Physics Subject Headings (PhySH)
Popular Summary
The ability to prepare a desired quantum state is essential for many areas of physics and technology, such as quantum computing, cold atom systems, and nuclear spin systems. However, finding optimal control sequences is a formidable challenge. Here, we adopt a radically different approach to this problem, based on machine learning. Using the same class of artificial intelligence algorithms that have recently yielded record-breaking performances on a variety of tasks such as Atari and Go, we show that cutting-edge reinforcement learning (RL) algorithms can learn to prepare a desired quantum state quickly and accurately.
Despite having no prior knowledge of quantum mechanics, our RL agent learns to prepare a system of coupled qubits (quantum bits) in a specified quantum state, using an iterative procedure and a single scalar reward as its only input. For a single-qubit system, the RL agent is even able to discover the underlying geometric structure of the quantum system—the Bloch sphere—despite having no direct knowledge that it is solving a quantum-mechanical problem.
Our work also highlights the difficulty of optimally preparing quantum states. We find that, generically, this is an extremely difficult task because small changes in the optimal protocol can result in a large difference in the prepared state. Nonetheless, we show that it is possible to find robust, near-optimal protocols to prepare many-body quantum states.
This work highlights the promise of RL for manipulating and controlling complex, out-of-equilibrium quantum systems and opens the possibility of directly coupling RL to experimentally realizable quantum-computing architectures.