Towards novel insights in lattice field theory with explainable machine learning

Machine learning has the potential to aid our understanding of phase structures in lattice quantum field theories through the statistical analysis of Monte Carlo samples. Available algorithms, in particular those based on deep learning, often demonstrate remarkable performance in the search for previously unidentified features, but tend to lack transparency if applied naively. To address these shortcomings, we propose representation learning in combination with interpretability methods as a framework for the identification of observables. More specifically, we investigate action parameter regression as a pretext task while using layer-wise relevance propagation (LRP) to identify the most important observables depending on the location in the phase diagram. The approach is put to work in the context of a scalar Yukawa model in $(2+1)\mathrm{d}$. First, we investigate a multilayer perceptron to determine an importance hierarchy of several predefined, standard observables. The method is then applied directly to the raw field configurations using a convolutional network, demonstrating the ability to reconstruct all order parameters from the learned filter weights. Based on our results, we argue that due to its broad applicability, attribution methods such as LRP could prove a useful and versatile tool in our search for new physical insights. In the case of the Yukawa model, it facilitates the construction of an observable that characterizes the symmetric phase.

Figure 1

Slice of the phase diagram for fixed Yukawa coupling $g=0.25$ using normalized values of $⟨M⟩$ and $⟨M_s⟩$. Phase transitions are highlighted by the shaded bars. We distinguish an antiferromagnetic (AFM), a paramagnetic (PM) and a ferromagnetic (FM) phase.

Figure 2

Sketch of LRP through the last two layers of a classification network that predicts one-hot vectors. Relevance is indicated by arrow width. The conservation law requires the sum of widths to remain constant during back propagation. Diagram adapted from [53].

Figure 3

Sketch of our strategy to learn meaningful structures from the simulation data by analysing the networks trained for action parameter inference. Field configurations used for training are either preprocessed into observables for the MLP (Approach A) or directly operated upon with a CNN (Approach B). Obtaining accurate predictions for the parameters indicates approximate cycle consistency in the above diagram, which supports the notion that the networks have successfully identified characteristic features. These can then be extracted in a subsequent interpretation step using LRP.

Figure 4

Results for the MLP. Top: predictions, bottom: normalized LRP relevances of individual features. Error bars here and throughout this work are obtained with the statistical jackknife method.

Figure 5

Benchmark results for the random forest. Top: predictions, bottom: nominal contributions of individual features.

Figure 6

Results for the CNN. Top: prediction, bottom: normalized relevances of individual filters. The dashed curve corresponds to the cumulative relevance of filter 3 and 4.

Figure 7

learnt weights of convolutional filters. Left to right: (no.1, PM); (no.2, AFM); (no.3, FM); (no.4, FM). The colour map is symmetric around zero. Red (blue) corresponds to positive (negative) weights.

Figure 8

Comparison of PM filters for three independent training runs of the CNN.

Figure 9

Normalized observables reconstructed from the learnt filters. The quantities associated with the FM and AFM phases are compared to $M$ and $M_s$. ${\mathcal{O}}_{\mathrm{PM}}$ and ${\mathcal{O}}_{\mathrm{PM}}^s$ are related by Eq. (6) and exhibit an approximate mirror symmetry around $\kappa =0$.

Figure 10

Convolutional filters corresponding to the observable ${\mathcal{O}}_{\mathrm{PM}}$ defined in Eq. (9) (left) and its corresponding staggered counterpart ${\mathcal{O}}_{\mathrm{PM}}^s$ (right).

Figure 11

Qualitative visualization of local structures in field configurations operated upon by the PM filter. Sign is encoded by arrow orientation/colour. Diagonal neighbors tend to share the same sign everywhere in the phase diagram. On the contrary, nearest neighbors show a preference toward either same (left) or opposite (right) orientations. ${\mathcal{O}}_{\mathrm{PM}}$ is particularly sensitive toward the local presence/absence of such sign flips in the PM phase, without the need for a globally nonzero expectation value of the magnetizations.