Setting Up Experimental Bell Tests with Reinforcement Learning

Finding optical setups producing measurement results with a targeted probability distribution is hard, as a priori the number of possible experimental implementations grows exponentially with the number of modes and the number of devices. To tackle this complexity, we introduce a method combining reinforcement learning and simulated annealing enabling the automated design of optical experiments producing results with the desired probability distributions. We illustrate the relevance of our method by applying it to a probability distribution favouring high violations of the Bell-Clauser-Horne-Shimony-Holt (CHSH) inequality. As a result, we propose new unintuitive experiments leading to higher Bell-CHSH inequality violations than the best currently known setups. Our method might positively impact the usefulness of photonic experiments for device-independent quantum information processing.

Figure 1

(a) Schematic representation of a Bell test with two parties—Alice and Bob. In each round, Alice and Bob independently perform a measurement on the state that they share. The measurement outcomes that Alice and Bob observe by repeating the measurements are used to test a Bell inequality. (b) Schematic representation of the proposed learning protocol to design photonic experiments leading to a probability distribution of measurement outcomes favoring a large CHSH inequality violation. Reinforcement learning (gray-green arrows) and simulated annealing (blue arrows) approaches are used together.

Figure 2

(a) Setup used to test the reliability of the simulated annealing based optimization. Two modes, initially empty, are coupled by a two-mode squeezing (TMS) operation. One mode is sent to each party which he or she measures using photon detection preceded by displacement operations (D). (b) Learning curves reflecting the parameter optimization process performance for the fixed sequence shown in (a). The results are an average over 1000 independent runs, and shaded areas are mean squared deviations. $\beta \approx 2.4547$ (dashed line) is the optimal CHSH value found by using an analytical expression. (c) CHSH score and experiment length reflecting the reinforcement learning process of improving the CHSH value using a restricted set of elements. The results are an average over ten independent runs, and shaded areas are mean squared deviations. The maximal value of $\beta \approx 2.7075$ found by the learning agent is shown as a dashed line. (d) Setup combining simplicity and a relatively high CHSH score $\beta \approx 2.6401$ (the heralding probability is $p_{\text{click}}\approx 2.2\times {10}^{-3}$). The diamond shape element above the heralding detector is a beam splitter.

Figure 3

Two photonic setups designed by the learning agent after learning from simpler examples. SMS elements correspond to single-mode squeezing operations. (a) A Bell test with CHSH score $\beta \approx 2.7242$ and a heralding probability $p_{\text{click}}\approx 2.9\times {10}^{-4}$, or $\beta \approx 2.7424$ with $p_{\text{click}}\approx 2.6\times {10}^{-5}$. (b) A Bell test with $\beta \approx 2.7454$ and $p_{\text{click}}\approx 1.1\times {10}^{-9}$. Parameters of the elements for both setups are in Supplemental Material [27]. (c) Dependence of the CHSH scores of setups which are considered in this work on detection efficiency. The setups designed by the agent provide higher CHSH values than known setups for any detection efficiency.