Parameter estimation using neural networks in the presence of detector effects

Histogram-based template fits are the main technique used for estimating parameters of high energy physics Monte Carlo generators. Parametrized neural network reweighting can be used to extend this fitting procedure to many dimensions and does not require binning. If the fit is to be performed using reconstructed data, then expensive detector simulations must be used for training the neural networks. We introduce a new two-level fitting approach that only requires one dataset with detector simulation and then a set of additional generation-level datasets without detector effects included. This simulation-level fit based on reweighting generator-level events with neural networks (SRGN) is demonstrated using simulated datasets for a variety of examples including a simple Gaussian random variable, parton shower tuning, and the top quark mass extraction.

Figure 1

An illustration of SRGN, applied to a set of synthetic and natural data. There is one synthetic dataset where the particle-level data (generation) is passed through a detector emulation (simulation). SRGN is a two-step process. First, a parametrized reweighting function is learnt using the generation dataset and a set of additional synthetic generator-level datasets. Second, the synthetic simulation is reweighted and compared with real data, iterated to converge on the parameters ${\theta }_{?}$. Illustration is inspired by Ref. [1].

Figure 2

A histogram of the Gaussian random variable $x$ for $\mu =0$ at geneator level and simulation level.

Figure 3

A demonstration of reweighting for generation (top) and simulation (bottom).

Figure 4

A demonstration of reweighting derived at generator level for the primary generator-level feature (top), the secondary generator-level feature (middle), and simulation level (bottom). In the top and bottom plots, a reweighting using only the primary generator feature is also shown (labeled ${\mathrm{DCTR}}_{\cancel{\mathrm{\Omega }}}$ wgt., where “wgt.” stands for “reweighted distribution”.)

Figure 5

For an individual run, the AUC versus $\mu$ for generator-level and simulation-level features. Also shown are the fitted values at both levels using scipy 1.5.2’s powell nondifferentiable optimizer fitting with 0.0001 as the relative error in both the solution ${\mu }_{\mathrm{SRGN}}$ and the function value $\mathrm{AUC}({\mu }_{\mathrm{SRGN}})$ acceptable for convergence, within the bounds [-2, 2] (the same range that the reweighting function is parametrized in), and optimizing for a maximum of 100 iterations.

Figure 6

An illustration of two-dimensional reweighting on generator level (top) and simulation level (bottom). For comparison, the results with analytical weights from Eq. (11) and the results with neural network-based weights are both plotted.

Figure 7

For an individual run, the AUC as a function of $\mu$ and $\sigma$. The true values are $\mu =-1.000$ and $\sigma =0.750$. The values of the nominal synthetic dataset are indicated by a green cross and the nondifferentiable optimizer’s fitted values are represented by a red cross.

Figure 8

One-dimensional fits for each of the three parton shower parameters. The vertical axes show the increase of the AUC for the classifier $g$ from its minimum value. For comparison, the ${\alpha }_s$ and strangeness plots also show fits using SRGN with only the inclusive or strange hadron multiplicity, respectively.

Figure 9

Features used to show the impact of generator parameter variations for the parton shower dataset. Variations in TimeShower:alphaSvalue, StringZ:aLund, and StringFlav:probStoUD are presented in the first, second, and third columns, respectively. Each row represents a different observable. Reweighted distributions are plotted over an average of 40 reweightings.

Figure 10

One-dimensional fits to the top quark mass. The vertical axes show the increase of the AUC for the classifier $g$ from its minimum value. The top plot uses all four observables and compares the fit at generator level to the fit at simulation level. The middle (bottom) is at generator level (simulation level) and compares the fit with all four observables to the fit with only $m_{b_1\mu \nu }$.

Figure 11

Histograms for the four observables for (left column) generator level for two different top quark masses, (middle column) a comparison of generator level and simulation level for a fixed top quark mass ($M_t=172.5\mathrm{GeV}$), and (right column) simulation level for two different top quark masses. Reweighted distributions are plotted over an average of 40 reweightings.

Figure 12

The distribution of fitted values ${\mu }_{\mathrm{SRGN}}$ for the simple one-dimensional Gaussian case over 120 runs, comparing the methods of maximizing loss and minimizing AUC.

Figure 13

Plotted here are the fully trained loss values of the classifying step for various values of StringZ:aLund, ensembled over 40 runs. It is clear that the loss is not maximized for the correct value of StringZ:aLund, 0.8000; conversely, AUC is (in comparison) smoothly minimized at the correct value (Fig. 8).