Machine Learning for Long-Distance Quantum Communication

Machine learning can help us in solving problems in the context of big-data analysis and classification, as well as in playing complex games such as Go. But can it also be used to find novel protocols and algorithms for applications such as large-scale quantum communication? Here we show that machine learning can be used to identify central quantum protocols, including teleportation, entanglement purification, and the quantum repeater. These schemes are of importance in long-distance quantum communication, and their discovery has shaped the field of quantum information processing. However, the usefulness of learning agents goes beyond the mere reproduction of known protocols; the same approach allows one to find improved solutions to long-distance communication problems, in particular when dealing with asymmetric situations where the channel noise and segment distance are nonuniform. Our findings are based on the use of projective simulation, a model of a learning agent that combines reinforcement learning and decision making in a physically motivated framework. The learning agent is provided with a universal gate set, and the desired task is specified via a reward scheme. From a technical perspective, the learning agent has to deal with stochastic environments and reactions. We utilize an idea reminiscent of hierarchical skill acquisition, where solutions to subproblems are learned and reused in the overall scheme. This is of particular importance in the development of long-distance communication schemes, and opens the way to using machine learning in the design and implementation of quantum networks.

Popular Summary

Machine learning has been successfully used for dealing with the vast amount of data that is generated and processed due to the ubiquitous use of technology. The quantum internet is potentially the next generation of a world-spanning network that uses the unique properties of quantum physics for information processing. In this work we use a machine learning agent for the task of connecting distant parties, which is one of the central services a quantum network would have to provide.

We show that a machine learning agent can learn a variety of existing protocols that are considered elementary for this task. Furthermore, we present methods for the use of machine learning as a tool to devise new schemes and show the advantages of our approach over other techniques.

Our approach establishes a new direction for exploring schemes for quantum information in an automated and autonomous fashion. It can become a practical tool to apply to new challenges without a rich spectrum of existing solutions such as the design of quantum networks.

Figure 1

Illustration of a reinforcement-learning agent interacting with the environment. The agent performs actions that change the state of the environment, while the environment communicates information about its state to the agent. The reward function is customized for each environment. ECM, episodic and compositional memory. The initial states for the different environments that we consider here are illustrated: (a) teleportation of an unknown state, (b) entanglement purification applied recurrently, (c) quantum repeater with entanglement purification and entanglement swapping, (d) scaling of quantum-repeater concepts to distribute long-distance entanglement.

Figure 2

Reinforcement learning of a teleportation protocol. (a) Initial setup: the agent is tasked to teleport the state of qubit $A^{\mathrm{\prime }}$ to $B$ using a Bell pair shared between $A$ and $B$. (b) Learning curves of an ensemble of PS agents: average number of actions performed in order to teleport an unknown quantum state when Clifford gates (magenta) and universal gates (blue) are the sets of available actions. (c) Two learning curves (magenta and blue) of two individual PS agents. Four solutions of different length are found by the agents. (d) Initial setup without predistributed entanglement. (e) Learning curve in the learning setting (d): average number of actions performed in order to teleport an unknown quantum state. In (b),(e), the curves represent an average over $500$ agents. The shaded areas show the mean squared deviation $\pm \sigma /3$. This deviation appears not only because of different individual histories of the agents, but also because of the difference in the individual solution lengths shown in (c).

Figure 3

Reinforcement learning of an entanglement-purification protocol. (a) Initial setup of the quantum communication environment: two entangled noisy pairs shared between two stations $A$ and $B$. (b) Cumulative reward obtained by $100$ agents for the protocols found after $5\times {10}^5$ trials. (c) Illustration of the best protocol found by an agent: Apply bilateral cnot operations and measure one of the pairs. If the measurement outcomes coincide, the protocol is considered successful, and $\sqrt{-iX}$ is applied to both remaining qubits before the next entanglement-purification step.

Figure 4

Reinforcement learning of a quantum-repeater protocol. (a) Initial setup for a length-2 quantum-repeater environment. The agent is provided with many copies of noisy Bell states with initial fidelities $F=0.75$, which can be purified on each link separately or connected via entanglement swapping at the middle station. (b) Learning curve in terms of resources (initial Bell pairs used) for the best of 128 agents with gate reliability parameter $p=0.99$. The known repeater solution (red line) is reached.

Figure 5

Reinforcement learning of a scalable quantum-repeater protocol. (a) Illustration of delegation of a block action. The main agent that is tasked with finding a solution to the large-scale problem (repeater length $4$ in this example) delegates the solution of a subsystem of length $2$ to another agent. That agent comes up with a solution to the smaller-scale problem, and that action sequence is then applied to the larger problem. This counts as one single action for the main agent. For settings even larger then this, the subagent itself can again delegate the solution of a subsubsystem to yet another agent. (b),(c) Scaling repeater with forced symmetric protocols with initial fidelities $F=0.75$. Gate reliability parameter $p=0.99$. The threshold fidelity for a successful solution is $0.9$. The red line corresponds to a solution with approximately $0.518\times {10}^8$ resources used. (b) Best solution found by an agent for repeater length 8. (c) Relative resources used by the agent’s solution compared with a symmetric strategy for different repeater lengths. (d),(e) Scaling repeater with asymmetric initial fidelities $(0.8,0.6,0.8,0.8,0.7,0.8,0.8,0.6)$. The threshold fidelity for a successful solution is $0.9$. (d) The best solution found by an agent with gate reliability $p=0.99$ outperforms a strategy that does not take the asymmetric nature of the initial state into account (red line). (e) Relative resources used by the agent’s solution compared with a symmetric strategy for different reliability parameters. (The jumps in the relative resources used are likely due to threshold effects or to agents converging to a very short sequence of block actions that is not optimal.)

Figure 6

Reinforcement learning of quantum-repeater protocols with imperfect memories. We consider an asymmetric setup with repeater length $8$ and initial fidelities $(0.8,0.6,0.8,0.8,0.7,0.8,0.8,0.6)$ arising from different distances between the repeater stations. Gate reliability parameter $p=0.99$. The threshold fidelity for a successful solution is $0.9$. (a) Learning curves (relative to the protocol designed for symmetric approaches with perfect memories) for different decoherence times $\tau$ of the quantum memories at the repeater stations. The dashed lines show the resources used by symmetric strategies that do not account for the asymmetric situation. (b) Resources (number of initial pairs) required by the protocols found by the agent for different memory times $\tau$ (relative to resources required for $\tau =\mathrm{\infty }$).

Figure 7

The PS network in its basic form, which corresponds to a weighed directed bipartite graph. Clips corresponding to eight states of the environment and seven possible actions are shown. They are connected by directed edges. The thick edges correspond to high probabilities of choosing action $a_k$ in the state $s_m$.

Figure 8

Entanglement-purification environment with automatic depolarization after each purification step. The figure shows the rewards obtained by 100 agents for the protocols found after $5\times {10}^5$ trials.

Figure 9

(a),(b) Scaling repeater with eight repeater links with symmetric initial fidelities of $0.7$. (a) Best solution found by an agent for gate reliability $p=0.99$. (b) Relative resources used by the agent’s solution compared with the working-fidelity strategy for different gate reliability parameters. (c),(d) Scaling repeater with eight repeater links with very asymmetric initial fidelities (0.95, 0.9, 0.6, 0.9, 0.95, 0.95, 0.9, 0.6). (c) Best solution found by an agent for gate reliability $p=0.99$. (d) Relative resources used by the agent’s solution compared with the working-fidelity strategy for different gate reliability parameters.