Deep reinforcement learning for complex evaluation of one-loop diagrams in quantum field theory

In this paper we present a technique based on deep reinforcement learning that allows for numerical analytic continuation of integrals that are often encountered in one-loop diagrams in quantum field theory. To extract certain quantities of two-point functions, such as spectral densities, mass poles or multiparticle thresholds, it is necessary to perform an analytic continuation of the correlator in question. At one-loop level in Euclidean space, this results in the necessity to deform the integration contour of the loop integral in the complex plane of the square of the loop momentum, to avoid nonanalyticities in the integration plane. Using a toy model for which an exact solution is known, we train a reinforcement learning agent to perform the required contour deformations. Our study shows great promise for an agent to be deployed in iterative numerical approaches used to compute nonperturbative two-point functions, such as the quark propagator Dyson-Schwinger equation, or more generally, Fredholm equations of the second kind, in the complex domain.

Figure 1

The parametrization of the branch cut through the angular variable $z$ turns out to be discontinuous for some values of $x$. The image shows an example of such a discontinuity, here for $x=4+i$. For this value of $x$, the branch cut Eq. (15) acquires the shape shown in the image. Starting at $z=-1$, with increasing value for $z$, the point $-\frac{1}{\sqrt{2}}-\epsilon$ is approached. At $z=-\frac{1}{\sqrt{2}}$, the parametrization jumps to the point $-\frac{1}{\sqrt{2}}+\epsilon$, and then approaches the first point from the other side, until it finally jumps back to the second point and runs towards $z=+1$, now tracing out the entire branch cut.

Figure 2

Properties observed by the reinforcement learning agent.

Figure 3

Two solutions provided by the same agent throughout training. Note that we do not apply any smoothing mechanisms, nor do we discourage or remove loops in the paths produced by the agent. Each contour starts at the origin and traces out a path that leads to the cutoff labeled $\mathrm{\Lambda }$. The (green) triangles mark the position of the poles, which were ignored in this particular run. The (yellow) circles between the cut's endpoints indicate the value of $x$ used for this particular run. The solutions presented here were produced by the best performing agent among the three trained in this setting. It has a success rate of almost $96\%$. Note that both contours solve the problem. Panel (a) shows a valid solution produced by the agent after training for 1783 episodes. Panel (b) shows a valid solution produced by the agent after training for 4726 episodes. This path could be post-processed to remove the loops, which we have not implemented.

Figure 4

This plot illustrates the development of averaged rewards throughout training of the best reference experiment, where the pole detection was deactivated. The blue curve represents the averaged return for each episode, while the green curve is the average over 100 consecutive averaged returns, rescaled for readability. The trend of improvement of the performance over training time is clearly visible.

Figure 5

Two solutions suggested by the same agent throughout training on the full problem (poles included). The (green) triangles mark the position of the poles. The solutions presented here were produced at a late time throughout training by the best performing agent among the 720 trained in this setting. It has a success rate of almost $28\%$, which is considerably lower than the one achieved without pole detection. Panel (a) shows a valid solution produced by the agent after training for 4996 episodes. Panel (b) shows an invalid solution produced by the agent after training for 4674 episodes. This contour must be rejected, because the path is not continuously deformable into the path along the positive real half-axis in absence of the branch cut.

Figure 6

This plot illustrates the development of averaged rewards throughout training of the best experiment with pole detection. The blue curve represents the averaged return for each episode, while the green curve is the average over 100 consecutive averaged returns, rescaled for readability.

Figure 7

Two solutions suggested by the same agent throughout training on the full problem (poles included) with autocompletion. The (green) triangles mark the position of the poles. The solutions presented here were produced by the best agent throughout training. It has a success rate of almost $48\%$, which is a dramatic increase in performance as compared to the best agent trained with naive pole detection. This indicates that this issue is due to a delayed reward signal. Panel (a) shows a valid solution produced by the agent after training for 126 episodes. Panel (b) shows a valid solution produced by the agent after training for 1820 episodes.