Beam-based rf station fault identification at the SLAC Linac Coherent Light Source

Accelerators produce too many signals for a small operations team to monitor in real time. In addition, many of these signals are only interpretable by subject matter experts with years of experience. As a result, changes in accelerator performance can require time-intensive consultations with experts to identify the underlying problem. Herein, we focus on a particular anomaly detection task for radio-frequency (rf) stations at the SLAC Linac Coherent Light Source (LCLS). The existing rf station diagnostics are bandwidth limited, resulting in slow, unreliable signals. As a result, anomaly detection is currently a manual process. We propose a beam-based method, identifying changes in the accelerator status using shot-to-shot data from the beam position monitoring system; by comparing the beam-based anomalies to data from rf stations, we identify the source of the change. We find that our proposed method can be fully automated while identifying more events with fewer false positives than the rf station diagnostics alone. Our automated fault identification system has been used to create a new dataset for investigating the interaction between the rf stations and accelerator performance.

Figure 1

Layout of the LCLS showing the location of the klystrons included in this work above the accelerating sections (blue rectangles: L0, L1, L2, and L3) and the names of the BPMs used to detect changes in beam energy: LTUH:250, LTUH:450, DMPH:502, and DMPH:693. The green triangles show dipoles and the red rectangles represent the free-electron laser undulators.

Figure 2

Two-stage rf station anomaly identification method.

Figure 3

Interpolated precision-recall curve for the BPM threshold on AMPL-based candidates. The best $F_1$ score is 0.897, at a BPM anomaly score threshold of 2.848, corresponding to an accuracy of 96%. The dashed line is the precision-recall curve for a random classifier. Note that recall here does not include anomalies missed by the AMPL threshold and is only an upper limit on the recall for the full algorithm.

Figure 4

Example of an rf station fault at KLYS:LI29:11. The top plot shows the klystron diagnostic data (both the AMM status bit and the AMPL signal). The middle plot shows the relative energy deviation (calculated from the beam position in the dispersive plane). The bottom plot shows the TMIT readings (which is the approximate number of electrons, subject to some calibration). The gray region defines the implied anomaly window, due to the delayed reporting of the klystron health information. This extreme example shows the AMPL drop nearly to 0, which causes the beam to be lost. The missing pulses in the energy deviation plot are the result of there being no beam to measure.

Figure 5

Example of a sustained anomaly at KLYS:LI28:11. The top plot shows the klystron diagnostic data (both the AMM status bit and the AMPL signal). The middle plot shows the relative energy deviation (calculated from the beam position in the dispersive plane), with a brief but large deviation followed by a recovery period. The bottom plot shows the TMIT readings (which is the approximate number of electrons, subject to some calibration). The gray region defines the implied anomaly window, due to the delayed reporting of the klystron health information. Note this anomaly is missed by AMM.

Figure 6

Count of anomaly candidates identified by either the AMM bit, the AMPL signal, or both that were labeled anomalies using our anomaly detection routine on the BPM data. The red part of the bar is anomaly candidates detected only by the AMPL signal, the blue part of the bar is anomaly candidates detected only by the AMM bit, and the magenta part of the bar is anomaly candidates that were detected by both. The klystrons are enumerated along the LCLS accelerator from low energy (left) to high energy (right).

Figure 7

Count of the number of anomalies detected by AMPL (red bars), the AMM bit (blue bars), and problems reported in CATER (black dashed lines) for various stations. The anomaly counts have been binned by week to produce the histograms. The shaded regions show weeks where the BPM data were available through the BSA service. Top: station 26-5. For this station, both the AMPL and AMM bit signals detect anomalies at approximately the same time. Middle: station 21-8. AMPL detects anomalous activity more often than the AMM bit does. Bottom: station 24-3. AMPL detects anomalous activity in April and November 2021 that the AMM bit misses entirely.

Figure 8

The model architecture of the demonstration DNN, showing the input size to each layer.

Figure 9

Interpolated precision-recall curve for the DNN classifier on our new rf station anomaly dataset. The dashed line is the precision-recall curve for a random classifier.