Parton shower uncertainties in jet substructure analyses with deep neural networks

Machine learning methods incorporating deep neural networks have been the subject of recent proposals for new hadronic resonance taggers. These methods require training on a data set produced by an event generator where the true class labels are known. However, this may bias the network towards learning features associated with the approximations to QCD used in that generator which are not present in real data. We therefore investigate the effects of variations in the modeling of the parton shower on the performance of deep neural network taggers using jet images from hadronic $W$ bosons at the LHC, including detector-related effects. By investigating network performance on samples from the Pythia, Herwig and Sherpa generators, we find differences of up to 50% in background rejection for fixed signal efficiency. We also introduce and study a method, which we dub zooming, for implementing scale invariance in neural-network-based taggers. We find that this leads to an improvement in performance across a wide range of jet transverse momenta. Our results emphasize the importance of gaining a detailed understanding of what aspects of jet physics these methods are exploiting.

Figure 1

We show the average jet images obtained for hadronic $W$ bosons and QCD as modeled by the PYTHIA default shower. The images on the top have been preprocessed in the standard way, while those on the bottom have also undergone the zooming procedure outlined in Sec. 3. The axes are left unlabeled since they do not correspond to the physical $\eta$ and $\varphi$ dimensions following image rotations, reflections, and zooming. Pixels are colored according to higher (red) and lower (blue) average normalized pixel intensities.

Figure 2

The ROC curves for the zoomed (solid blue) and unzoomed (dashed green) jet images for the PYTHIA default shower. The lower panel shows the ratio of the zoomed to unzoomed efficiencies, also showing the efficiency sliced in bins from $200<p_T<300\mathrm{GeV}$ (dotted-dashed red) and $300<p_T<500\mathrm{GeV}$ (short dashed blue).

Figure 3

This figure shows the W-jet image differences between the default PYTHIA shower and the alternate VINCIA shower in PYTHIA (top left), the default SHERPA shower (top right), the default HERWIG angular shower (bottom left) and the HERWIG dipole shower (bottom right). The plots have been individually normalized.

Figure 4

This figure shows the ROC curves of the PYTHIA (solid blue), VINCIA (dashed green), HERWIG angular (red dotted-dashed) and dipole (dashed purple), and SHERPA (solid gold) showers for the DNN output (left) and the combination of the jet mass and $n$-subjettiness ratio ${\tau }_{21}$ through a two-dimensional binned likelihood ratio (right). The lower panels show the ratio of the ROCs with the default PYTHIA shower. All ROC curves are computed using jet images within a window on the jet mass, $50<m<110\mathrm{GeV}$, and transverse momentum, $250<p_T<300\mathrm{GeV}$.