Uncertainty aware anomaly detection to predict errant beam pulses in the Oak Ridge Spallation Neutron Source accelerator

High-power particle accelerators are complex machines with thousands of pieces of equipment that are frequently running at the cutting edge of technology. In order to improve the day-to-day operations and maximize the delivery of the science, new analytical techniques are being explored for anomaly detection, classification, and prognostications. As such, we describe the application of an uncertainty aware Machine Learning method using the Siamese neural network model to predict upcoming errant beam pulses using the data from a single monitoring device. By predicting the upcoming failure, we can stop the accelerator before damage occurs. We describe the accelerator operation, related Machine Learning research, the prediction performance required to abort the beam while maintaining operations, the monitoring device and its data, and the uncertainty aware Siamese method and its results. These results show that the researched method can be applied to improve accelerator operations.

Figure 1

Setup of the differential current monitor showing beam passing by the sensors and through the SCL.

Figure 2

Beam macropulse pattern and beam current waveform.

Figure 3

The deterministic Siamese model architecture (a) is composed of two input layers for the reference waveforms and the acquired waveform, followed by one common ResNet model and a distance function. Then a dropout layer is applied to avoid overfitting, and finally an output layer to compress the dimensionality and produce the similarity scalar. The uncertainty aware model (b) replaces the output layer with a Gaussian process approximation layer in order to produce both the similarity and the uncertainty values.

Figure 4

Digitized waveform of the beam current before the errant beam pulse and the errant beam pulse itself.

Figure 5

Box plot displaying a random sample of 20 waveforms from the training set. The orange horizontal line corresponds to the median, the upper (lower) edge of the whisker corresponds to the maximum (minimum) value, and the upper (lower) edge of the box denotes the third (first) quantiles.

Figure 6

Example results using scipy.signal.find_peaks function with a minimum height of 2 mA and minimum distance of 75 elements between neighboring peaks.

Figure 7

Classifier output histogram (the subplot represents a zoomed-in version of the same histogram plot).

Figure 8

Deterministic model ROC curves for train and test samples.

Figure 9

Model predicted uncertainty versus uncertainty aware model prediction on a test data set. The blue dots are the normal pulses, the orange dots are the anomaly type used for training, and the red dots are the anomalous pulses that the model was not trained on.

Figure 10

Uncertainty aware model ROC curves for train and test samples. Dark error bands are for the 25%/75% quantiles and the light color bands are for the 10%/90% values.

Figure 11

ROC curve with uncertainty band for the inferences made on anomaly type 1111 with the model trained on anomaly type 1100. The dark and light bands represent the same range as Fig. 10.

Figure 12

Block diagram of a generic Convolutional AutoEncoder architecture.

Figure 13

Performance comparison of multiple Convolutional Autoencoder architectures against the proposed SNN. The horizontal axis is the maximum TPR below the predetermined FPR threshold, while the vertical axis represents model size in terms of the number of trainable parameters.