Measurement-based adaptation protocol with quantum reinforcement learning

Machine learning employs dynamical algorithms that mimic the human capacity to learn, where the reinforcement learning ones are among the most similar to humans in this respect. On the other hand, adaptability is an essential aspect to perform any task efficiently in a changing environment, and it is fundamental for many purposes, such as natural selection. Here, we propose an algorithm based on successive measurements to adapt one quantum state to a reference unknown state, in the sense of achieving maximum overlap. The protocol naturally provides many identical copies of the reference state, such that in each measurement iteration more information about it is obtained. In our protocol, we consider a system composed of three parts, the “environment” system, which provides the reference state copies; the register, which is an auxiliary subsystem that interacts with the environment to acquire information from it; and the agent, which corresponds to the quantum state that is adapted by digital feedback with input corresponding to the outcome of the measurements on the register. With this proposal we can achieve an average fidelity between the environment and the agent of more than $90\%$ with less than 30 iterations of the protocol. In addition, we extend the formalism to $d$-dimensional states, reaching an average fidelity of around $80\%$ in less than 400 iterations for $d=11$, for a variety of genuinely quantum and semiclassical states. This work paves the way for the development of quantum reinforcement learning protocols using quantum data and for the future deployment of semiautonomous quantum systems.

Figure 1

(a) Bloch representation of the environment state at the initial time, with $|0\rangle$ being the state of the agent. (b) Bloch representation of the environment in the $j\mathrm{th}$ iteration, where $|{\overline{0}}_j\rangle$ is the state of the agent, which was rotated in the previous iterations.

Figure 2

Quantum circuit diagram for the measurement-based adaptation protocol. The box labeled with M indicates the projective measurement process, and the red lines denote feedback loops.

Figure 3

Performance of the measurement-based adaptation protocol for the qubit case. Panel (a) shows the mean fidelity for 2000 initial random states, and panel (b) shows the value of ${\mathrm{\Delta }}^{\left(j\right)}$ in each iteration.

Figure 4

Performance of the measurement-based adaptation protocol for a total random state given by Eq. (15) with $d=11$. Panel (a) shows the mean fidelity for 2000 initial random states, and panel (b) shows the value of ${\mathrm{\Delta }}_j$ in each iteration.

Figure 5

Performance of the measurement-based adaptation protocol for a coherent state of the form in Eq. (16). Panel (a) shows the mean fidelity for 2000 random pairs $\{a,b\}\in [0,1)$, where $\alpha =a+ib$, and panel (b) shows the value of ${\mathrm{\Delta }}_j$ in each iteration.

Figure 6

Performance of the measurement-based adaptation protocol for genuinely quantum states. Panel (a) shows the mean fidelity for 2000 cat states of the form in Eq. (17), and panel (b) shows the mean fidelity for 2000 repetitions of the protocol using the superposition ${\left(\right|0\rangle }_E+{|10\rangle }_E)/\sqrt{2}$ for the environment.