• Open Access

Reinforcement Learning in Different Phases of Quantum Control

Marin Bukov, Alexandre G. R. Day, Dries Sels, Phillip Weinberg, Anatoli Polkovnikov, and Pankaj Mehta
Phys. Rev. X 8, 031086 – Published 27 September 2018
PDFHTMLExport Citation

Abstract

The ability to prepare a physical system in a desired quantum state is central to many areas of physics such as nuclear magnetic resonance, cold atoms, and quantum computing. Yet, preparing states quickly and with high fidelity remains a formidable challenge. In this work, we implement cutting-edge reinforcement learning (RL) techniques and show that their performance is comparable to optimal control methods in the task of finding short, high-fidelity driving protocol from an initial to a target state in nonintegrable many-body quantum systems of interacting qubits. RL methods learn about the underlying physical system solely through a single scalar reward (the fidelity of the resulting state) calculated from numerical simulations of the physical system. We further show that quantum-state manipulation viewed as an optimization problem exhibits a spin-glass-like phase transition in the space of protocols as a function of the protocol duration. Our RL-aided approach helps identify variational protocols with nearly optimal fidelity, even in the glassy phase, where optimal state manipulation is exponentially hard. This study highlights the potential usefulness of RL for applications in out-of-equilibrium quantum physics.

  • Figure
  • Figure
  • Figure
  • Figure
  • Figure
  • Figure
  • Figure
  • Received 12 January 2018
  • Revised 1 August 2018

DOI:https://doi.org/10.1103/PhysRevX.8.031086

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society

Physics Subject Headings (PhySH)

Condensed Matter, Materials & Applied PhysicsStatistical Physics & ThermodynamicsQuantum Information, Science & TechnologyAtomic, Molecular & OpticalInterdisciplinary PhysicsGeneral Physics

Authors & Affiliations

Marin Bukov1,*, Alexandre G. R. Day1,†, Dries Sels1,2, Phillip Weinberg1, Anatoli Polkovnikov1, and Pankaj Mehta1

  • 1Department of Physics, Boston University, 590 Commonwealth Avenue, Boston, Massachusetts 02215, USA
  • 2Theory of quantum and complex systems, Universiteit Antwerpen, B-2610 Antwerpen, Belgium

  • *mbukov@bu.edu
  • agrday@bu.edu

Popular Summary

The ability to prepare a desired quantum state is essential for many areas of physics and technology, such as quantum computing, cold atom systems, and nuclear spin systems. However, finding optimal control sequences is a formidable challenge. Here, we adopt a radically different approach to this problem, based on machine learning. Using the same class of artificial intelligence algorithms that have recently yielded record-breaking performances on a variety of tasks such as Atari and Go, we show that cutting-edge reinforcement learning (RL) algorithms can learn to prepare a desired quantum state quickly and accurately.

Despite having no prior knowledge of quantum mechanics, our RL agent learns to prepare a system of coupled qubits (quantum bits) in a specified quantum state, using an iterative procedure and a single scalar reward as its only input. For a single-qubit system, the RL agent is even able to discover the underlying geometric structure of the quantum system—the Bloch sphere—despite having no direct knowledge that it is solving a quantum-mechanical problem.

Our work also highlights the difficulty of optimally preparing quantum states. We find that, generically, this is an extremely difficult task because small changes in the optimal protocol can result in a large difference in the prepared state. Nonetheless, we show that it is possible to find robust, near-optimal protocols to prepare many-body quantum states.

This work highlights the promise of RL for manipulating and controlling complex, out-of-equilibrium quantum systems and opens the possibility of directly coupling RL to experimentally realizable quantum-computing architectures.

Key Image

Article Text

Click to Expand

Supplemental Material

Click to Expand

References

Click to Expand
Issue

Vol. 8, Iss. 3 — July - September 2018

Subject Areas
Reuse & Permissions
Author publication services for translation and copyediting assistance advertisement

Authorization Required


×
×

Images

×

Sign up to receive regular email alerts from Physical Review X

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

×

Log In

Cancel
×

Search


Article Lookup

Paste a citation or DOI

Enter a citation
×