Sparse autoregressive models for scalable generation of sparse images in particle physics

Generation of simulated data is essential for data analysis in particle physics, but current Monte Carlo methods are very computationally expensive. Deep-learning-based generative models have successfully generated simulated data at lower cost, but struggle when the data are very sparse. We introduce a novel deep sparse autoregressive model (SARM) that explicitly learns the sparseness of the data with a tractable likelihood, making it more stable and interpretable when compared to generative adversarial networks (GANs) and other methods. In two case studies, we compare SARM to a GAN model and a nonsparse autoregressive model. As a quantitative measure of performance, we compute the Wasserstein distance ($W_p$) between the distributions of physical quantities calculated on the generated images and on the training images. In the first study, featuring images of jets in which 90% of the pixels are zero valued, SARM produces images with $W_p$ scores that are 24%–52% better than the scores obtained with other state-of-the-art generative models. In the second study, on calorimeter images in the vicinity of muons where 98% of the pixels are zero valued, SARM produces images with $W_p$ scores that are 66%–68% better. Similar observations made with other metrics confirm the usefulness of SARM for sparse data in particle physics.

Figure 1

Calorimeter images in particle physics are often very sparse, where most of their pixels have very small values. Left panel: typical signal image of a hadronic jet from [6]. Right panel: typical signal image of the vicinity of a muon from [7].

Figure 2

Pixel generation process by a deep ARM to create an image with $D$ pixels. For the pixel $x_i$, a deep neural network (DNN) is evaluated on a vector with values $x_0,\dots ,x_{i-1}$, zero padded to length $D$. The output of the network is the parameters ${\theta }_i$ of a parametric probability density $P(x_i|{\theta }_i)$, from which $x_i$ is sampled.

Figure 3

Generation process of a deep autoregressive model. During generation, the first pixel $x_0$ is sampled from $x_0\sim P(x_0|{\theta }_0)$. Next, the pixel $x_0$ is zero padded to a $D$-dimensional vector and passed to the neural network ARM model, which evaluates the parameters $\mathit{\theta }=\{{\theta }_0,\dots ,{\theta }_{D-1}\}$, though only ${\theta }_1$ is needed to sample the next pixel $x_1\sim P(x_1|{\theta }_1)$. The pixels $x_0$ and $x_1$ are again zero padded to create a $D$-dimensional vector which is passed into the neural network to generate the next pixel. This process is repeated until all pixels are generated. Note that the same neural network is used at each generation step, and part of its weight connections are disabled to preserve the autoregressive structure.

Figure 4

Generation process for the $\mathrm{D}+\mathrm{C}$ model. The blue circle dots represent the value sampled for each pixel. For example, given the first pixel value of 6.7, sampled from the empirical distribution of the dataset, the neural network outputs the distribution parameters ${\gamma }_1=0.1$, ${\mu }_1=3.1$, $s_1=3.9$ to generate the second pixel. Then a Bernoulli random variable is sampled from $z_1\sim \text{Bern}({\gamma }_i)$ and a logistic random variable is sampled from ${\tilde{x}}_i\sim \text{Logistic}({\mu }_i,s_i)$. The value of the second pixel $x_i$ is produced by the product of these two variables as $x_i=z_i·{\tilde{x}}_i=0·2.9=0$. This sequential process is repeated until every pixel is generated.

Figure 5

Left panel: Distribution of pixel values in the jet substructure dataset for the nine pixels in the center of the images (central region) and the rest of the pixels (peripheral region). Note that the majority of the pixels in the peripheral region are zero valued and in general have lower variance than pixels in the central region. Right panel: Two-stage generation for the central and peripheral regions using a spiral path and two different SARM modules. Using different networks for each region improves performance.

Figure 6

Example of jet images generated from each model. Notice that SARM-2 ($\mathrm{D}+\mathrm{C}$) is able to produce small value pixels in the periphery of the images. The intensity of each pixel is shown on a log scale, where the white space represents pixels with value zero.

Figure 7

Pixelwise average of the images generated by each model. Notice that lagan struggles to capture the distribution of low value pixels in the periphery of the images and has a nonsmooth radial transition compared to the autoregressive models. The intensity of each pixel is shown on a log scale, where the white space represents pixels with value zero.

Figure 8

Distributions of jet observables (top panels: mass, bottom panels: $P_{\mathrm{T}}$) calculated from images generated by several generative models and from the original images generated by pythia. Signal images, with two collimated quarks, are on the left; background images, with a single quark or gluon, are on the right.

Figure 9

Evaluation of the fidelity of images generated by several models in the context of a classification task. Images generated by the model are used to train a network to discriminate between signal and background, but performance is measured using the original pythia images.

Figure 10

Example calorimeter images in the vicinity of a muon from the generative models as well as the original Monte Carlo generator. The top row shows isolated muons (signal), while the bottom shows muons produced in association with a jet (background). The intensity of each pixel is shown on a log scale, where the white space represents pixels with value zero.

Figure 11

Pixelwise averages of calorimeter images in the vicinity of a muon from the generative models as well as the original Monte Carlo generator. The top row shows isolated muons (signal), where little calorimeter activity is expected. The bottom row shows muons produced in association with a jet (background), which deposits significant energy near the muon. A linear scale is used to reveal the differences between signal and background images.

Figure 12

Distributions of calorimeter observables (top panels: invariant mass, bottom panels: total $P_{\mathrm{T}}$) calculated from images generated by several generative models and the originals generated by a Monte Carlo generator. Signal images, in the vicinity of an isolated muon, are on the left. Background images, in the vicinity of a muon produced with an associated jet, are on the right.

Figure 13

Evaluation of the fidelity of images generated by several models in the context of a classification task, distinguishing muons produced in isolation from those produced in association with a jet. Images generated by the model are used to train a network to discriminate between signal and background, but performance is measured using the original Monte Carlo images.

Figure 14

Left panel: density plot of training data. Right panel: density plot of generated data. The two distributions are very close, showing that the ARM is able to learn the joint distribution of $x_0$ and $x_1$ well.

Figure 15

Error measured by subtracting of the pixelwise average of the images created by each generative model and the pixelwise average of the images generated with pythia. The SARM models have lower error than both Pixel $\mathrm{CNN}++$ and lagan with most of the errors are concentrated in the center of the image.

Figure 16

Distribution of aggregated pixel intensity in the generated images for jet substructure study. Notice most of the differences happen at high pixel values where there are fewer events. lagan also has a harder time replicating the distribution of background images across all pixel values compared to the other models.

Figure 17

Typical muon images generated using lagan. The figures are plotted in log scale, where the white space represents pixels with value zero.

Figure 18

Pixelwise average of muon images from lagan for signal and background. The average images generated by lagan fail to reproduce the radial structure present in the average Monte Carlo images (Fig. 11).

Figure 19

Comparison of the mass and $P_{\mathrm{T}}$ distributions of the images generated by lagan, SARM-2 ($\mathrm{D}+\mathrm{D}$), and the Monte Carlo simulations for both signal and background muons.

Figure 20

Subtraction between the pixelwise average of generated images vs Monte Carlo images. The errors are evenly distributed in the signal images, while they are concentrated in the center for the background images. In the center there is larger number of high intensity pixels.

Figure 21

Distribution of pixel intensity for muon isolation study. Pixel $\mathrm{CNN}++$ underrepresents the distribution while the SARM models miss the high pixel values where there are fewer events.

Figure 22

Additional typical images from the jet substructure study.

Figure 23

Additional typical images from the muon isolation study.