Performance in the vertical test of the 832 nine-cell 1 . 3 GHz cavities for the European X-ray Free Electron Laser

The successful production and associated vertical testing of over 800 superconducting 1.3 GHz accelerating cavities for the European X-ray Free Electron Laser (XFEL) represents the culmination of over 20 years of superconducting radio-frequency R&D. The cavity production took place at two industrial vendors under the shared responsibility of INFNMilano–LASA and DESY. Average vertical testing rates at DESYexceeded 10 cavities per week, peaking at up to 15 cavities per week. The cavities sent for cryomodule assembly at Commissariat à l’énergie atomique (CEA) Saclay achieved an average maximum gradient of approximately 33 MV=m, reducing to∼30 MV=mwhen the operational specifications on quality factor (Q) and field emission were included (the so-called usable gradient). Only 16% of the cavities required an additional surface retreatment to recover their low performance (usable gradient less than 20 MV=m). These cavities were predominantly limited by excessive field emission for which a simple high pressurewater rinse (HPR)was sufficient. Approximately 16%of the cavities also received an additionalHPR, e.g. due tovacuum problems before or during the tests or other reasons, but these were not directly related to gradient performance. The in-depth statistical analyses presented in this report have revealed several features of the series produced cavities.


I. INTRODUCTION
The successful construction of the 17.5 GeV European X-ray Free Electron Laser (XFEL) [1] represents the culmination of over 20 years of superconducting radiofrequency (SRF) R&D.The core high-gradient 1.3 GHz nine-cell niobium cavity and cryomodule technology was originally developed by an international collaboration coordinated by DESY for TeV-Energy Superconducting Linear Accelerator (TESLA)-a 500 GeV center-of-mass electron-positron linear collider and integrated x-ray FEL, formally proposed in 2003 [2].The primary goal of this development was the reduction in cost per MVover the then existing SRF accelerators by a factor of 20.This was successfully achieved by an increase in gradient by over a factor of 5 to an average of approximately 30 MV=m, the remaining factor of 4 accruing from the integration of eight cavities into a single cryogenic cryomodule.A total of 102 cryomodules containing 816 cavities have been produced, of which 97 have been installed in the SRF linacs of the European XFEL.This represents by far the largest deployment of TESLA technology to date.
Preparations for the industrial cavity production began in 2006.The foreseen production and testing rates of eight cavities per week represented a significant step up from any previous series production.TESLA-type elliptical niobium cavities had already been successfully mechanically fabricated by industry during the R&D phase, but the final and delicate surface chemical treatments required for high performance had always been made by the labs.For the European XFEL series production, the cavities were completely constructed by industry, including mechanical fabrication and chemical surface preparation, and were delivered to DESY ready for cold vertical testing.This required significant technology transfer to industry; the two vendors contracted for the cavity production [Ettore Zanon S.p.a.(EZ), Italy, and RI Research Instruments GmbH (RI), Germany] had to invest in the construction and commissioning of significant new infrastructure in order to meet the stringent fabrication requirements and production schedule.
Cavity delivery began in December 2012, ramping up to full production rate in October 2013, and then continued until the end of 2015 [3].The niobium material was procured by DESY and supplied to the vendors [4].A key production strategy was the so-called "build to print" concept, whereby the vendors where required to extensively document each step of the production, adhering to well-defined procedures and tolerances.Providing the cavities were fabricated to these specifications, DESY accepted the cavity without any final cold rf performance guarantee.As a result, DESY took the responsibility of any remedial (postproduction) measures necessary to recover poor-performing cavities.
The resulting average performance of ∼30 MV=m with an unloaded Q value Q 0 ≥ 10 10 of such a large industrial series production represents an unprecedented success for this SRF technology.
Such a large scale production can only be successfully completed by the work of a large team of experts in various fields.The responsibility for cavity production was shared between INFN Milano-LASA and DESY and included about 25 people in the core team.Cold vertical testing of individual cavities as well as cryomodule tests were performed at DESY in the Accelerator Module Test Facility (AMTF) [5] by a team of 30 technicians, engineers, and physicists from the Henryk Niewodniczański Institute of Nuclear Physics Polish Academy of Sciences (IFJ-PAN) Krakow supported by DESY experts.
This report focuses on the detailed analysis of the resulting performance of the series production cavities.The main body of the report is separated into the following four sections: Section II, Cavity Production, provides a brief overview of the mechanical fabrication and chemical surface treatment of the cavities by industry, and introduces some key concepts and terminology important for the remainder of the report.
Section III, Vertical Acceptance Test, details the setup and procedures for the cold vertical testing of the cavities in AMTF.This section introduces the overall workflow and in particular the definition of the usable gradient, and provides details of the careful procedures and quality checks-quality assurance (QA) and qauality control (QC)-of the cavities during this testing period.The observed nonconformities during QC and cold vertical testing are also briefly discussed.
Section IV, Production Performance Statistics, provides a comprehensive set of statistical analyses which quantify the overall cavity production in terms of gradient, Q 0 , and field-emission performance.Performance limitations and the impact of the surface retreatments performed at DESY on (predominantly) low-performing cavities are also quantified, and performance trends during the production discussed.Finally the results from the European XFEL series production are compared to existing historical data.
Section V, Detailed Cavity Performance Studies, covers the following special-topic studies: differences between vendor performance; impact of the infrastructure at both the vendors and at DESY; studies on high pressure water rinse (HPR) application; impact of special procedures such as transport, helium-tank welding, repair of the two-phase helium pipe; studies of process and field-emission onset during the vertical test; and an estimate of the residual surface resistance based on additional 1.8 K measurements.
Finally, Sec.VI provides a summary and some concluding remarks.

II. CAVITY PRODUCTION
The cavity design for the European XFEL accelerator cavity is based on the "TESLA cavity design" [6] as a superconducting nine-cell cavity made of solid niobium with the π-mode frequency of the TM010-passband at 1300 MHz.Minor modifications compared to the original design have been applied for a simpler fabrication and cost reduction [7].

A. Cavity fabrication and material
Series production of the 1.3 GHz cavities was equally divided between Ettore Zanon S.p.A., and RI Research Instruments GmbH.Production included both mechanical fabrication and surface preparation [3,8] together with the required extensive quality assurance, quality management, and associated documentation [9].Details about the niobium and niobium-titanium material used can be found in [4].The Nb material for the accelerating cells was delivered in differing amounts by three companies (Tokyo Denkai, 1 Ningxia OTIC, 2 and SE Plansee 3 ).The material from each company was distributed equally to the two cavity vendors.The rf measurements for quality assurance during the cavity production are described in [10].
Table I provides a summary of the cavity production numbers.A total of 852 cavities were produced by the vendors (429 and 423 cavities by RI and EZ, respectively).This number comprises the original order of 840 cavities total, plus an additional eight cavities as replacements for rejected cavities (at the vendors' expense), together with an additional order of four cavities.Of these cavities, 831 have vertical tests which are included in the analysis (see Sec. IV).804 XFEL series cavities (401 by EZ; 403 by RI) were delivered complete with helium tank (Fig. 1), ready for vertical testing at DESY in AMTF.Each vendor also produced an additional 12 cavities without a helium tank for the ILC-HiGrade program [11,12], which were used as a quality control tool as well as for further R&D.For 8 of these 24 cavities a subsequent assembly of the He tank was made.In addition, 4 of the additional 16 cavities used for infrastructure setup and commissioning at the vendors [3] have since been fitted with a He tank for use in the assembly of the 102 cryomodules.The concept of the delivery "ready for vertical testing" included transportation under vacuum conditions.All cavities were fully equipped with their Higher-Order Mode (HOM) antennas, pick-up probe, a high-Q input coupler antenna with a fixed coupling, and two beam-tube flanges.The high-Q input coupler was only required for the vertical test and was subsequently removed prior to module assembly, while the remaining HOM antennas and pick-up probe remained in place.
As noted in the introduction, both vendors were required to follow exactly well-defined specifications for the mechanical fabrication and surface treatments, but no cold rf performance guarantee was required [3].An important risk DESY accepted was writing and enforcing the precise and complete specification.A key result of this approach was that DESY ultimately accepted the risk of lowperformance cavities together with the responsibility for any necessary mitigation (i.e.surface retreatment, see Sec.IV D), providing the cavities were shown to have fulfilled the build to print specifications.

B. Surface preparation
The surface preparation at both vendors (Fig. 2) [3,8,[13][14][15][16] started with a bulk electropolishing (EP; EZ: 140 μm, RI: 110 μm) followed by 800 °C annealing, but for the final surface polishing two alternative recipes were used: EZ applied a final chemical surface removal ("flash BCP" where BCP stands for buffered chemical polishing) of 10 μm, while RI applied a final EP of 40 μm ("final EP").Both final preparation steps included extensive rinsing with ultrapure water at a pressure of ∼100 bar in an ISO 4 cleanroom (HPR), particle free assembly of all flanges under ISO 4 cleanroom conditions, well-defined vacuum requirements, and a 120 °C baking.
All cavity transports-from the vendor to DESYas well as from DESY to CEA Saclay-were made by truck with the cavity orientated horizontally in specially designed transport boxes for minimizing mechanical shocks to the cavity [17].The cavities were transported under vacuum without active monitoring.The vacuum status was part of the incoming inspection at both DESY and CEA Saclay.Data loggers for shocks were used during the transport from DESY to CEA Saclay for one cavity of each batch and on the truck itself.
For those cavities requiring an additional surface treatment, the surface reprocessing (either at DESY or at the vendors) was strictly performed under the same cleanliness requirements as for the initial surface preparation (see Secs.III A, IV D, and V F).The choice of the retreatment procedure applied depended on the characteristics of the observed limitation in the cold vertical test and was decided on a cavity-by-cavity basis.Three main retreatment procedures were applied: HPR: After the cold vertical test the cavity first had its outer surface cleaned before being brought into the ISO4 cleanroom, where it was then vented with ultrapure, particle-free nitrogen before undergoing six standard HPR of 2 h each.For the HPR, only the beam tube flange with the angle valve was removed and all other flanges and antennas were kept on the cavity.The beam tube flange with the angle valve was reassembled after HPR.The cavity was then pumped, leak checked (including residual gas analysis), and transported back to the vertical test stand.
BCP: After cleaning, venting, and disassembly, the cavity received a 10 μm chemical surface removal (BCP) followed by several rinsing steps and one standard 2-h HPR.After reassembly and leak check, the equipped cavity received six standard HPR of 2 h each.The cavity was then leak checked (including residual gas analysis), received a 120 °C baking outside of the cleanroom, and was transported to the vertical test stand.
Grinding: Precondition for grinding was an optical inspection at OBACHT (optical bench for automated cavity inspection with high resolution on short time scales) [18,19] performed after the vertical test in order to identify the surface defects4 to be ground.Local grinding only took place at EZ [20].Afterwards the residual artefacts from the grinding were removed by a 20 μm BCP and the surface quality was restored by a 20 μm EP (or more, depending on the ground defect), followed by water and ethanol rinsing.The removal of the defects was confirmed by performing a further optical inspection at EZ.The frequency and field flatness were then checked and, if necessary, corrected to the required values.Finally BCP was applied as described above.

III. VERTICAL ACCEPTANCE TEST
To verify conformity with the specification for manufacture and the criteria for module assembly, several test procedures were defined which were applied between arrival at DESY of the cavities from the manufacturer up to the preparation for shipment to the cryomodule assembly plant at CEA Saclay.The complete vertical acceptance test sequence for cavities is divided into three main parts: incoming inspection, cold vertical test, and outgoing inspection.

A. Overview of vertical acceptance test workflow
The vertical acceptance test in the AMTF [5,21] was on one hand used as a tool for quality control of the cavity production, and on the other hand to facilitate sorting of cavities of similar performance into cryomodules.An overview of the workflow is shown in Fig. 3.A series cavity with no nonconformity (NC) and with acceptable rf performance (see Sec. III B) followed the standard FIG. 2. Overview of the surface treatment at RI (final EP scheme, left) and EZ (final flash BCP scheme, right).workflow (white marked boxes) starting from "incoming inspection" to "rf test (VT)" and on to the "outgoing inspection."Upon review, each vertical test (VT) received one of several possible "decisions" (see Fig. 3); with the decision "send to Saclay" (point 5 in Fig. 3) the cavity was accepted for cryomodule string assembly and shipped to CEA Saclay."ILC HiGrade" cavities without NC were internally used for further QA and R&D purposes after the vertical test was complete (Fig. 3, point 4).
In the case of NC at key points in the workflow, different retest or retreatment cycles (or both) were applied as follows: A NC during the incoming inspection (Sec.III C 1) first underwent an expert review and was then either rejected and sent back to the vendor (Fig. 3, point 3) or repaired at DESY, depending on the reason for the NC.
Handling of vacuum and rf NC during the cold vertical test itself (Fig. 3, point 2) are described in Sec.III D.
An unacceptably low usable gradient (see Sec. III B) in the vertical test triggered a retreatment cycle (Fig. 3, point 1).As there was no performance guarantee required of the vendor, performance-driven retreatments were in general made at DESY. 5 The choice of retreatment process was decided by an expert review, and depended on the critical limitation of the rf test as well as on the history of the cavity.In most cases a cycle of HPR was applied as the first retreatment (see Secs.II B and IV D).
The special case of cavities being returned from CEA Saclay is briefly discussed in Sec.III E.
The results of each incoming inspection and outgoing inspection were documented in a report stored in the DESY Engineering Data Management System (EDMS) [9,22].A well-defined data set of relevant rf parameters from each valid cold vertical test was stored in the European XFEL cavity database (see Sec. III G).

B. Definition of usable gradient and acceptance criteria
Although all cavities were tested to their maximum achievable gradient (E max ), of greater importance for accelerator operation was the usable gradient (E usable ), which took Q 0 as well as field-emission performance into account.It was defined as the lowest value of (i) quench gradient (quench limited), (ii) gradient at which Q 0 drops below 10 10 (Q 0 limited), and (iii) gradient at which either of the x-ray detectors exceeds the threshold (field-emission limited).
For the field-emission limit, the acceptable x-ray thresholds were set to 0.01 mGy=min and 0.12 mGy=min for the top and bottom detector (see Fig. 4), respectively.The threshold 0.01 mGy=min was based on experience from the cavity testing for free-electron laser in Hamburg (FLASH).The higher limit for the lower detector was a geometrical effect.A bottom detector was implemented for the first time at DESY in order not to miss downwards directed radiation and to get a more accurate value for the usable gradient.For about 9% of the cold vertical tests, the bottom detector limited the usable gradient.
At the beginning of production, the criterion for acceptance for module assembly was specified as E usable ≥ 26 MV=m, chosen to give a margin of ∼10% compared to the required average design operational gradient (23.6 MV=m at Q 0 ≥ 10 10 ).Based on an analysis of about 270 cavities tested up to May 2014, including the necessary retreatments and retests, the acceptance threshold was reduced to E usable ≥ 20 MV=m, in order to optimize the number of vertical tests while still maintaining an average module gradient of ≥23.6 MV=m.
Cavities with E usable < 20 MV=m were considered for further processing or re-treatment.The exact nature of the handling of low-performance cavities was judged on a case-by-case basis (expert review, see Sec.III A).

Quality control checks
An important tool for quality control for all European XFEL cavities was the incoming inspection, which was the first step after the cavity was received at DESY from the vendor, and was used to detect nonconformities from the specification.Immediately after arrival, a brief visual inspection of the cavity within the transport box (Sec.II B) was made to look for obvious damage, followed by the mechanical part of the incoming inspection after removing the cavity from its transport box.Each cavity was checked for completeness, correct assembly of all components, as well as for mechanical NC, such as significant scratches, dents, and other irregularities.More than 500 cavities received an additional dimension check of the mechanical longitudinal axis, which was compared to the data provided by the vendors.
In order to detect possible deformations of the cavity shape during transportation, the frequency spectrum of the TM010 passband was compared to the last one measured at the vendor before shipping.In addition, the electrical integrity of the rf antennas was checked.
After passing these checks the cavity was installed into a test insert [23] ready for the cold vertical test (see Sec. III C 2).After mechanical installation into the insert, the cavity was connected to a pumping unit and a leak check and residual gas analysis (RGA) were performed (the vacuum part of the incoming inspection, see Sec.III C 5).
All observations were recorded in the so-called "incoming inspection report." When the cavity failed to pass one or more of the criteria of the incoming inspection and an ad hoc repair was not possible, it was sent back to the vendor for repair.Table II gives the total number of cavities which were returned to the vendors from the 832 cavities that received a regular incoming inspection.Approximately 6% of all cavities failed to pass incoming inspection, mostly in the first months of the series production.The main reasons for failing incoming inspection were wrong torque on the accessories screws, inconsistent data for the mechanical axis, or vacuum problems including visible contaminations ("fibers") on the inside of the cavity angle valve.An additional 11% of the cavities were sent back after the cold vertical test.The reasons included cavities with vacuum problems due to poor assembly by the vendor (bad sealing) or cavities with surface irregularities observed on the inner surface [18,19].
Finally, after the vertical test and before being shipped for cryomodule assembly, an outgoing inspection was performed.It included a detailed visual outside inspection as well as additional measurements of the TM010 passband spectrum and a check of the HOM rejection filter tuning.(The vacuum integrity had already been addressed during the leak check and RGA made directly after the cold vertical test; see Sec.III C 5).The results were recorded in an outgoing inspection report.If the cavity passed the outgoing inspection, it was released for cryomodule assembly and shipment to CEA Saclay.

Vertical test stands and rf procedures
The vertical acceptance tests of the series as well as the HiGrade cavities took place in the AMTF at DESY [24,25].In order to achieve the desired testing rate of at least eight to ten cavities per week, the cold vertical tests were made using two independent test systems (labeled XATC1 and XATC2), each consisting of an independent bath cryostat (Fig. 4) and rf test stand.Each test cryostat accepts one "insert" which supports up to four cavities, greatly increasing the efficiency of cool-down and warm-up cycles.Six inserts were available to fulfill the required testing rate (see Fig. 5).As all cavities were tested with assembled HOM feedthroughs, the cold vertical tests used a "long pulse" mode with typically 5-20 s rf-on followed by an rf-off time of approximately 50 s in order to protect the HOM feedthroughs against overheating during the Q 0 ðEÞ measurements described below.A true continuous wave (CW) operation would have increased the heat load on the feedthroughs by a factor of 100 compared to the nominal European XFEL rf operation mode.In addition, the damping properties of the HOM notch filters were such that only Q 0 ðEÞ measurements for the π mode of the TM010 passband were possible.
All cavities were equipped with fixed rf input antennas (high Q antenna).For 2 K operation the expected unloaded Q value (Q 0 ) of the cavities was about 2 × 10 10 .The design value of the external input Q value (Q ext ) was 8 × 10 9 for an effective coupling at higher gradients in order to allow rf processing in case of field emission (see Sec. IV B 3).The experimentally observed Q ext typically varied between 5 × 10 9 and 9 × 10 9 , while the cavity probe showed a typical coupling factor of <0.01.The external Q values of both HOM couplers were adjusted by tuning of the HOM notch filters before each vertical test [26].
The cold vertical tests followed a standardized and automated procedure, which included the measurement of the Q 0 value versus the accelerating gradient (E) at 2 K for the fundamental π mode, as well as the frequencies of all modes of the TM010 passband. 6For each point of the Q 0 ðEÞ curve, x rays were measured inside the concrete shielding above and below the cryostat (Fig. 4).For each vertical test at least two power rises were performed and compared with respect to the rf parameters and cavity performance.Only after achieving two consecutive repeatable Q 0 ðEÞ curves was the vertical test considered complete.For each power rise, the operator attributed a limiting reason for the maximum achieved gradient according to one of the following categories: breakdown, breakdown with field emission, forward power limited, forward power limited with field emission, or quenching caused by heating of the HOM coupler.
In order to avoid frequent damage of the rf input cables the maximum rf input power was limited to about 200 W. Beyond this no general administrative limit was applied, but the surrounding of the cryostats was under permanent radiation protection surveillance.Radio-frequency cables and connections were checked for defects before each test by time domain reflectivity measurement.
In addition to the Q 0 ðEÞ curves, many cavities had the HOM frequencies and Q 0 values of the TE111, TM110, and TM011 modes measured [27].
For the series vertical tests in AMTF, no additional diagnostics like T-mapping, second sound, or fluxgate magnetometer were available.A few cavities obtained additional vertical tests for diagnostics purposes in a different test facility [19,28,29].
The vertical test infrastructure started operation in February 2013 and achieved full operation in October 2013, with a stable average testing rate of approximately 40 vertical tests per month (Sec.IVA).The last vertical acceptance test was finished in March 2016.

Ambient magnetic field in the test cryostats
The cryogenic losses for superconducting rf accelerators depend linearly on the Q 0 value of the resonators.One significant influence on the Q 0 (more specifically the rf surface resistance) is the ambient magnetic field.Hence a suppression of the ambient magnetic field below a value of 1-2 μT is necessary (see Chap. 3.4.2 in [30], [31,32]).
The magnetic field in the concrete pit for the two vertical cryostats was approximately 40 μT.Simulations showed that two magnetic shields were necessary: a double-walled warm magnetic shield around the cryostats (300 K), and an additional smaller one inside of the cryostat at 2 K.All shields were cylindrical with a closed bottom (see Fig. 4).Unfortunately there was no space available to use the more effective geometry of a smaller double-walled cold magnetic shield.The outer warm shields cover the complete cryostat except the top.They were constructed from μ-metal, which has its maximum permeability close to room temperature.The cold magnetic shield, placed inside of the cryostat, was made out of Cryoperm™, a material with a higher permeability at cryogenic temperatures [33].Its height above the top of the cavities was limited to 270 mm due to the position of helium supply and return pipes.All shields obtained a heat treatment (up to 1100 °C) after manufacturing (cutting, drilling, and welding) to bring the materials back to their initial magnetic properties [34].Measurements of the magnetic field inside of the cryostat were made at room temperature and are given in Fig. 6.

Cool-down procedures
The cool-down procedures at the AMTF vertical test stands were heavily automated (programmable logic controller based) and only required operator supervision [21,35].Cernox™ temperature sensors on one of the inserts were used to commission the procedures and monitor the mechanical stress on the inserts [23].Carbon temperature sensors (TVO) [36] glued to the outside of the cryostats were used during regular operation.During series vertical testing, neither the inserts nor the individual cavities were equipped with temperature sensors.
The cool down from 300 to 100 K took approximately 12 h, which represents an average cool-down rate of 5 mK=s.The cryostat remained at 100 K for 4-6 h to enable a test for hydrogen Q disease (see Chap. 3.4.1 in [30]), after which the cavity was further cooled to 4 K.The cool-down rate across T c was of the order of 200 mK=s, at which point a maximal longitudinal temperature gradient along the cavities of about 25 K was typically observed.
The final cool down to 2 K was performed manually by reducing the vapor pressure of the helium bath to ∼30 mbar, achieving an average cool-down rate of 0.5 mK=s.

Vacuum systems
Fundamental to the cold-vertical-test philosophy for the series production was the concept that the cavities remained under vacuum from the vendor to the cryomodule string assembly at CEA Saclay.However, it was considered beneficial to allow active pumping on the cavity vacuum during vertical testing, to provide continuous monitoring of the vacuum status and in particular to quickly identify leaks when they occurred during the tests.This required the cavity to be openly connected to a local pumping system during the test, which needed to be performed in a carefully controlled fashion so as not to compromise the vacuum of the cavity.The fundamental requirements with respect to leak rate and RGA are described in [37].
Figure 7 shows the vacuum schematic of the insert used in the tests.The vacuum for each of the four cavities was effectively kept independent, so that if there was a vacuum issue with one cavity, this cavity could be actively pumped and the remaining three could still be tested.The four valves labeled V4.1-V4.4 were the all-metal angle valves that were delivered with the equipped cavity from the vendor (one per cavity).The vertical test insert had its own set of all-metal angle valves (labeled V3.1-V3.4)located at the top plate of the test insert, which are connected to the corresponding valves on the cavities via pipes with bellows.This arrangement allows the insert vacuum system to be individually monitored and pumped to the required vacuum conditions at both warm and cryogenic conditions.The vacuum-related procedure for a vertical test was as follows: 1.The four cavities (as delivered from the vendor, i.e. under vacuum with their angle valves closed) were mounted into the insert in the preparation area and connected to the insert vacuum system via the four bellows.The insert vacuum was then pumped (up to the cavity angle valves) using the local pump station in the preparation area (via the main valve, labeled V1 þ V2 in Fig. 7), and the vacuum status monitored, including RGA.In general all four pipes were pumped in parallel, and a common RGA made to assure the insert vacuum had achieved the required specification. 7.Once the required vacuum of ≤10 −7 mbar was achieved in the insert connections, the cavity angle valves were opened one by one.According to [37], for each individual cavity the leak rate had to be below 10 −10 mbar l=s at a pressure inside the cavity of ≤10 −7 mbar (typically after 30 min of pumping) and the RGA requirements had to be satisfied.After each vacuum check the cavity angle valve was closed before the next cavity angle valve was opened.After a successful check of all cavities, all cavity angle valves were reopened.
3. The main valves V1, V2 and the insert angle valves (V3.1-V3.4)were then closed and the entire insert plus cavities disconnected from the pumping station in the preparation area.The loaded insert was then transported across to one of the two vertical test cryostats where it was reconnected to the test-stand pumping station.After the specified vacuum of ≤10 −7 mbar was achieved on the pump-side vacuum system, all valves were opened again to allow a common RGA measurement, after which the vacuum gauges (FRG 1-4) were switched on.When the pressure was below the requisite 10 −7 mbar, the insert angle valves were closed.
4. During the entire thermal cycle the pressure was constantly monitored.If the pressure increased in a cavity, the insert angle valve connected to that cavity was opened for active pumping.If despite active pumping the pressure exceeded 10 −2 mbar, the cool-down cycle had to be aborted.
5. After warm up to room temperature and after closing all insert angle valves and the main valves, the insert was disconnected from the test-stand pump station and transported back to the preparation area.
6.A final common RGA and leak check was then made for all cavities, after which the cavity angle valves where closed, the insert vacuum lines vented, and the cavities disassembled from the insert.
All vacuum connections were made in a particle-free environment using local clean rooms (ISO 5) to minimize the risk of contamination or leaks, and an oil-and particlefree state for all the vacuum parts was mandatory [37].
The design of the insert vacuum system is based on DN40 pipes which provide sufficient pumping speed for the cavities.As noted above, with the exception of the need to actively pump a cavity, the insert angle valves V3.1-V3.4 were closed during the test and subsequent warm up.As a safety measure against overpressure during the warm up (ensuing from a cold liquid helium leak during the test), four burst discs were located on the top of the insert between the warm insert valves and cold cavity valves.

Measurement errors
A bottoms-up analytical estimate based on the usual rf test equations ( [38,39], see Chap. 8 in [40]) results in an uncertainty of independent rf measurements (test to test) of about ∼10% for E and up to ∼20% for Q 0 .This takes into account the reproducibility of cable connections, errors of the test devices (e.g.power meter, frequency counter, and oscilloscope) as well as the finite directivity of the directional couplers.Within a single cold vertical test and each Q 0 ðEÞ curve the observed measurement scatter is much smaller (about 1% and 3% for E and Q 0 , respectively) and only depends on the errors (e.g.resolution and drift effects) of the test devices.
An additional statistical error analysis has been estimated from the distributions of the cable calibration parameters and the external Q (Q ext ), based on about 800 cold vertical tests [41].The resulting total rms error from these effects is 6.6% for Q 0 measurement and 3.3% for the gradient measurement.These errors are well within the analytical estimates above, and as they do not include all effects, the more conservative values were generally taken.
The estimated uncertainty in the usable gradient (see Sec. III B) was in general larger, since its determination was based on interpolation of either the field-emission (x ray) or the Q 0 curves during the power rise.In the event that the local derivative of either of these values with respect to the measured gradient was small, the error in the respective value would naturally transform into a larger uncertainty in usable gradient.This was particularly true of Q 0 -limited cavities, where the slope around the Q 0 ¼ 10 10 threshold was often relatively low.

D. Nonconformities during vertical test
The cold vertical tests are not without risk and nonconformities did occur.The main NC were either vacuum problems (cold leaks, leaks or damage on accessories, or vacuum problems with the test setup), or problems occurring during the rf measurements.Both cases will be considered separately below.

Vacuum problems
Leak tightness of the superconducting rf cavities under cryogenic conditions was an essential requirement for assembly into a cryomodule.Hence a leak check was done before and after the cold vertical test.During the cold vertical test the vacuum pressure was observed and recorded.If a deviation in the pressure to the standard behavior was detected during the cool-down or warm-up cycle, the test was declared as nonconforming.If the leak could be localized, the defect component was repaired or exchanged.If not, the cavity was then reconnected to the vacuum system and tested again, in order to exclude that the problem arose from the vacuum connection between the cavity and the insert (see Sec. III C 5).If this test was successful (i.e.no leak seen), then the cavity was released for the next step.In the case where the second test still indicated the presence of a leak, it was generally concluded that the leak was due to the cavity itself, and all components (antennas and flanges) were then disassembled and reassembled under cleanroom conditions followed by a full HPR cycle, after which a further vertical test was performed to check the performance (Sec.IV D).
Table III gives a breakdown of all vacuum NC during the vertical acceptance tests.No Q 0 ðEÞ measurements were possible for 86 vertical tests due to a vacuum NC in one or more of the four cavities in the same insert (see Sec. III C 5).All cavities could be successfully recovered and eventually fulfilled the vacuum requirements according to the specifications.

Radio-frequency-related NC
In 24 cold vertical tests a problem with the rf measurement itself was observed and it was decided to repeat the test to verify the performance of the cavity.At the beginning of the series cavity testing, several minor hardand software problems with the new infrastructure at AMTF occurred; this included saturated analog-to-digital converter's (solved by correct adjustment of rf attenuators) and phase locking problems (solved by an additional phase switch).Over the 4 years of cold rf testing, a few broken rf input power cables required a retest.In some cases, an unusual coupling parameter value or some other strange rf behavior was observed which could not be clarified immediately during the cold vertical test.In these cases an investigation took place after warm up and the cold vertical test was repeated.

E. Cavities returned from CEA Saclay
A total of 52 cavities were returned to DESY after being shipped to the cryomodule assembly facility at CEA Saclay.The reasons were due to either nonconformities during string assembly (for example, incorrect venting or pumping procedures or coupler or gate valve assembly errors), nonconformities identified during the incoming inspection at CEA Saclay (vacuum leaks, defective valves, etc.), or cavities recalled by DESY (additional testing, repairs, or transport tests).Most (but not all) of these cavities required retreatment (HPR) and cold vertical testing.All the cavities successfully achieved the usable gradient requirements and were subsequently returned to CEA Saclay.The over 60 vertical tests associated with these special cases are not included in the analysis presented in Sec.IV.

F. Categorization of tests
To facilitate detailed statistical analysis of the over 1200 vertical test results, a series of flags were developed to categorize each individual test, based on the reason why that test was performed.The vertical tests were broadly divided into the following categories: the primary acceptance test performed on receipt of the cavity from the vendors (referred to as "as received" tests); a vertical test after an additional surface treatment ("retreatment"); repeated tests ("retests"), in general due to a technical issue with the test itself; and a relatively small fraction of tests ("other") which were associated with commissioning of infrastructure, or were performed on cavities before integration into their helium tanks at the request of the vendors (referred to as "preliminary acceptance tests").Where applicable, the tests were further categorized by where the associated action (e.g.retreatment) occurred, i.e. either at DESY or at the vendor.Finally, to facilitate more detailed-level analysis, the retreatment and retest categories were further subdivided into the exact reasons for the test.
For the tests flagged as retreatment related, the type of retreatment was also recorded e.g.high-pressure rinse only (HPR), buffered chemical polishing (BCP) followed by HPR, and a 120 °C bake.
The analysis results presented in Sec.IV are extensively based on the use of the flags to filter the cavity test results.However, to simplify the analysis, only the primary level of categorization was used (as received, retreatment, retest, or other) with the notable exception of retreatment, where two analysis-level subcategories were defined: retreatment (performance) and retreatment (other).The distinction is important since it allows us to cleanly separate out those vertical test results which are associated with a retreatment applied in order to increase the rf performance of a cavity (recovery), from retreatments which were necessary due to some NC with the vertical test itself (for example, a vacuum leak during the test; see Sec.III D for details).
Tests flagged as "as received" are particularly noteworthy.Ideally, all cavities should have had an "as received" test as the first test after acceptance of the cavity from the vendor.However, approximately 10% of the cavities do not have an "as received" test, due to the fact that these cavities had a nonstandard production history, and in particular already underwent an additional surface retreatment before the first vertical test.The set of "as received" tests is considered to give the best indication of the production performance for cavities having undergone only the specified production process.See Sec.IV for more details, and also Sec.III C for information concerning NC during incoming inspection.
A complete summary table of the database flags is given in the Appendix.

Database for AMTF processes
The AMTF data base [42] (based on Oracle™ RDBMS 8 ) was created to document (i) the progress and status of work performed on cavity and accelerator modules including a time stamp and the name of the responsible operator, (ii) the results of rf tests in warm and cold conditions, (iii) the check and release of key rf data of each vertical acceptance test by the responsible expert as an additional QC before transferring them to the European XFEL cavity database, (iv) the usage of infrastructure at AMTF, (v) the "decision" made after the vertical acceptance test defining the following workflow of a cavity (see Fig. 3), and (vi) the physical location of cavities and cryomodules.
The AMTF database was a mandatory and effective tool for organizing the daily workflow in AMTF, handling multiple cavity and cryomodule activities in parallel.

The European XFEL cavity database
The so-called "European XFEL cavity database" [43][44][45][46] is also based on the Oracle RDBMS platform and contains selected information about the fabrication, treatment, and testing of cavities, high-power input couplers, and cryomodules.
For the cavity fabrication, key data were transferred to the database mainly by an interface to the DESY EDMS system, but also by direct data loading, especially for rf measurement data.At AMTF the TM010 passband frequencies, the characteristic data of the vertical acceptance test (Sec.III C 2) as well as HOM frequencies of dedicated modes were stored.Data loading from the AMTF database to the European XFEL database was done automatically based on flagging new data for loading.
In addition, for the vertical acceptance tests, the European XFEL database included a special graphical user interface-based editor to allow an expert to manually add or modify the following database entries: (i) a flag to categorize the reason of each individual test (Sec.III F), (ii) description of the cavity treatment before the test, (iii) correction of the x-ray thresholds with respect to the usable gradient for each power run of the test, (iv) correction of the measured Q 0 parameter, (v) general comment for each test, and (vi) cavity retreatment procedure description as a standardized abbreviation.
This information together with the calculation of the "usable gradient," Q 0 ð4 MV=mÞ, Q 0 ð23.6 MV=mÞ, for each power run were extensively used for the analysis presented in Sec.IV.A significant effort was undertaken to ensure the consistency of the database data by manual crosschecking with the raw data.
A subset of the vertical acceptance test data can be accessed by a graphical user interface, providing for example individual Q 0 ðEÞ curves, listing key rf data, and comparing TM010 passband frequencies of either individual or several selected cavities.The direct access to all data requires Oracle SQL, which allows the further analysis with tools like Mathematica™ as applied in this publication.

IV. PRODUCTION PERFORMANCE STATISTICS
In this section the results of the vertical tests and the effectiveness of the surface retreatments will be discussed.Here the focus is on the overall statistical figures of merit (mean, rms, yield, etc.) of the cavities produced, and the dominant performance limitations.Section V will deal with more specific specialized studies.

A. Cold vertical test statistics
All of the 831 cavities used in the analysis (Table IV) received one or more cold vertical tests.The total number of cold vertical tests was approximately 1340, of which 1200 have been used in this analysis.The remaining 10% of the tests were either "preliminary acceptance" tests of cavities requested by the vendors (before being mounted into the helium tank), tests rejected due to technical issues with the tests themselves, special tests used for infrastructure commissioning, or finally tests associated with the 52 cavities that were returned from the cryomodule assembly facility at CEA Saclay due to some NC during the string assembly (see Sec. III E).
Ignoring rejected and excluded tests, the average number of vertical tests per cavity was approximately 1.48 (1.45 and 1.52 for RI and EZ, respectively).This includes all necessary additional testing due to surface re-treatment of cavities or for other test-related reasons.
Figure 8 shows the number of cold vertical tests per calendar month in the AMTF.The average total testing rate is indicated by the dashed line (taken from 1 Januray 2013 to 31 October 2015), and is approximately 41.4 tests per month, corresponding to roughly 10.4 tests per week.However much higher testing rates were achieved as can be seen from the figure, corresponding to peaks of up to 15 tests per week.
Figure 9 shows the fraction of the 831 cavities used in the analysis which have exactly 1, 2, etc. cold vertical tests recorded in the database.
The reasons that a cavity required an additional cold vertical test can be loosely divided into two categories: performance related, i.e. the usable gradient of the cavity was considered unacceptable and the cavity was sent for additional surface treatment and a subsequent cold vertical test; or nonperformance related, were the cavity was retested due to some technical issue with the cavity or the test itself (see Secs.III D and III E). Figure 10 shows a breakdown of the reasons for cavities having a second cold vertical test as an example.
Performance related tests are those labeled "retreatment at DESY (performance)" and "retreatment at vendor (performance)," the latter representing cavities that were sent back to either RI or EZ for rework."Retreatment at DESY (other)" and "retreatment at vendor (other)" represent nonperformance-related tests where some additional surface retreatment was still required (generally HPR), for example  in the case of a vacuum leak during the cold vertical test.
"Retest" indicates a simple repeat of the cold vertical test, usually due to some rf problems, or check of cavity after the 2-phase pipe repair or some transport issue.Finally, we consider the breakdown of the first cold vertical test in the database and the important concept of the "as received" test (see Sec. III F).All cavities received at least one cold vertical test at DESY, of which approximately 90% were categorized as "as received."The remaining 10% of cavities had either undergone an additional retreatment at the vendor or in a few cases at DESY before this first cold vertical test, in general due to problems with incoming inspection as outlined in Sec.III C. Figure 11 shows the breakdown of the first cold vertical test in the database for the 831 cavities.

B. "As received" yields
In the following, the primary statistics for the "as received" performance of the European XFEL cavity production are presented.As explained above, this corresponds to approximately 90% of all the cavities in the analysis.However, cold vertical tests categorized as "as received" were considered the best indication of the results of the nominal production process, since this excluded cavities which required additional treatment due to identified nonconformities or other nonperformance related issues before being cold vertical tested at DESY.
Figure 12 shows the distribution and yield curves for the maximum and usable gradients achieved in the "as received" cold vertical tests.
The average maximum and usable "as received" gradient is 31.4MV=m and 27.7 MV=m respectively, with a corresponding rms spread of 6.8 MV=m (22%) and 7.2 MV=m (26%).Thus approximately 4 MV=m (12%) is lost from the maximum measured performance due to either above-threshold field emission or low-Q 0 behavior.For the production, the more important figure of merit is the usable gradient yield above the performance acceptance criteria for European XFEL cryomodule assembly, originally set to 26 MV=m and then later dropped to 20 MV=m.Table V summarizes the key figures of merit for both the maximum and the usable gradient.
It is important to note that only approximately 50% of the cavities were limited by quench (breakdown), of which ∼12% also showed some field emission.Of the remaining cavities, approximately 22% (35% if field-emitting cavities are included) were rf-power limited in the cold vertical test, suggesting that their true physical quench limit is higher.However, the rf-power-limited tests were predominantly caused by a low Q 0 as a result of high-field Q slope, resulting in Q 0 values that were below the acceptable threshold.Figure 13 shows the maximum achieved gradient and the associated Q 0 for all "as received" cold vertical tests, broken down by test-limit reason.The low Q 0 values of the rf-power-limited cavities (orange) can be clearly seen.The remaining 16% of cavities were limited in the test due to other technical reasons including excessive heating of the HOM coupler 9 or in a few cases technical limits caused by problems of the rf test stand.
Of more interest for European XFEL is the breakdown of the limiting criteria for the usable gradient as shown in Fig. 14.The large fraction (51%) of cavities limited by only Q 0 (no FE)-i.e. with no field emission-reflects the similar fraction of cavities having achieved gradients greater than 30 MV=m, where the high-field Q slope FIG.12.The distributions and yield curves for the maximum achieved gradient (blue) and the usable gradient (orange) for all cavities "as received."Yield is defined as those cavities which have a gradient greater than or equal to the specified value.(The darker color represents the overlap of the two histograms.)was the predominant limiting effect.The other half of the cavities was limited by breakdown (20%) or field emission (29%), where for the latter the 9% of tests limited by Q 0 indicated in Fig. 14 are included, since these tests also showed strong indications of field emission which is assumed to be the underlying cause of the observed low Q 0 .
Figure 15 shows the distribution of usable gradient in the "as received" cold vertical tests with cavities limited by field emission separated out (including the 9% Q 0 -limited cases indicated in Fig. 14).Although the non-FE limited distribution does show a long tail down to low gradients, the dominant limiting factor below approximately 24 MV=m is field emission, while above it is mainly either breakdown or Q 0 .Figure 16 shows the non-FE distribution (as shown in Fig. 15) separated into breakdown (BD) and Q 0 .Here the dominance of the Q 0 limit is evident above 20 MV=m.The low-gradient tail is mostly BD although several poor-performing cavities showed Q 0 below 10 10 at gradients less than 10 MV=m.
Figure 17 shows the distribution of Q 0 measured at low gradient (4 MV=m), generally field emission free, and at 23.6 MV=m (nominal design gradient).The majority of measurements exceed the requirement of 10 10 except for a small fraction at 23.6 MV=m (∼6%).The average performance is 2.1 × 10 10 and 1.3 × 10 10 for 4 MV=m and 23.6 MV=m, respectively, clearly indicating the drop in Q 0 performance with higher gradient.
Although there was predominantly no field emission observed in the cavities at 4 MV=m, this is not true for 23.6 MV=m (nominal design gradient) as indicated in Fig. 15. Figure 18 shows the Q 0 results at 23.6 MV=m separated into those cold vertical tests showing field emission and no field emission.For the cavities showing field emission, it is not possible to determine if the Q 0 is limited by that field emission, or represents the fundamental Q 0 of the cavity (comparable to the no-field emission case).However, below 10 10 field emission is likely the dominant mechanism.

C. Production performance trends
In the previous section the key "as received" performance statistics for the complete cavity production was presented.In this section the history of the performance over the approximately three-year production period is discussed.Again, the "as received" cold vertical test performance will be presented, as a function of the delivery date to DESY. 10  Figure 19 shows the production history binned by month for the "as received" usable gradient as a boxwhisker chart.The two horizontal red dashed lines indicate the average for the first and second half of production.An overall improvement in the average gradient can be seen in the second half.
The overall improvement can be further seen in the improvement in yield above 20 MV=m (leading to an overall reduction in the number of retreatments and subsequent vertical tests).Figure 20 shows the number of cavities with "as received" usable gradient in the ranges <20 MV=m, 20-26 MV=m, and 26-28 MV=m.Cavities with usable gradients ≥28 MV=m are not explicitly shown but can be inferred.The data is calculated for quarter-period production.A marked reduction in the number of cavities with usable gradient below 20 MV=m can be clearly seen   This differs slightly from the definition used in [3], where shipment date from the vendor was used.
in the second half of production, leading to an overall rejection rate of approximately ∼10%, as compared to ∼20% during the first production period.
Finally, for completeness Fig. 21 shows the monthly "as received" production statistics for Q 0 , measured at a gradient of 4 MV=m.The average low-field Q 0 remained relatively constant over the entire production period at approximately 2.1 × 10 10 , and unlike the usable gradient, no discernible improvement in the second half of production was seen.

D. Surface retreatment
As already noted, an important aspect of the European XFEL procurement contracts was that there were no requirements specified for vertical acceptance test performance: the so-called "build to print" vendor specifications [3] meant that it was the responsibility of DESY to provide remedial measures to improve poor performance cavities.
As discussed in Sec.III B, the original acceptance threshold for the usable gradient was set to 26 MV=m (European XFEL nominal design gradient plus 10%), but this was then relaxed to 20 MV=m in May 2014, essentially to reduce the number of additional vertical tests associated with retreatments, while keeping the required average module gradient.
A total of 313 retreatments 11 were performed for 237 of the 831 cavities (∼29%) included in the analysis (an average of 1.3 retreatments per cavity).Of these retreatments, roughly two-thirds were performed in the DESY infrastructure while the remaining third was performed at the vendors. 12However not all of these retreatments were due to the performance criteria discussed above and in Sec.III B: Approximately 49% of the retreatments (44% at DESY, 5% at vendor) were a direct result of nonacceptable rf performance in the vertical test, corresponding to approximately 16% of the cavities.The remaining 51% of the retreatments (22% at DESY, 29% at vendor, also corresponding to ∼16% of the cavities) were necessary as a result of some nonconformance with either the cavity (Sec.III C 1) or the test itself (Sec.III D), e.g. a vacuum leak, which subsequently required an additional surface treatment (in the majority of cases an additional HPR).About 3% of the cavities got surface retreatments due to both performance criteria and nonconformance of the cavity or the test.
For 16% of the cavities which required retreatment due to performance limitation, the exact choice of retreatment applied depended on the details of the observed performance in the cold vertical test, and was decided on a cavity-by-cavity basis.Of these, a standard HPR cycle was by far the most applied (∼86% of the cases); approximately 9% underwent BCP followed by HPR and 120 °C bake, while in a few cases (∼4%) mechanical grinding [20] was required (in general after an optical inspection [18,19]; see Sec.V F) followed by further surface treatment as necessary.
In general, a cavity whose performance in the cold vertical test was considered unacceptable (i.e. the usable gradient was below 20 MV=m) was first retreated with HPR.This proved to be very effective in increasing the usable gradient, in particular for those cavities whose performance was field-emission limited.Figure 22 shows the breakdown of the reasons for the first retreatment at DESY (16% of the total number of cavities in the analysis; see Fig. 10), clearly showing that field emission dominates the reason for the performance-driven retreatments.
In general, cavities were sent for BCP (Sec.II B) if, after an additional HPR, their usable gradient was still below 20 MV=m.Several of the cavities which repeatedly resulted in an unacceptably low usable gradient were sent for optical inspection in OBACHT in order to identify the reason of the poor performance, and subsequent remedial action was decided depending on the result.
Figure 23 shows the fraction of cavities undergoing subsequent retreatments at both DESY and the vendors.Approximately 15% of cavities required an initial performance-driven retreatment after the first vertical test, dropping to ∼4% for a second retreatment.Less than 3% of the cavities required a third retreatment.
Figure 24 shows the usable gradient before and after retreatment for both HPR and BCP retreatments.usable gradients below 20 MV=m (Fig. 25), ∼70% are recovered to above 20 MV=m with just over half being above 26 MV=m.The resulting average is 25.5 MV=m.A similar improvement can be seen for the application of BCP (although the statistics are much lower).
The application of HPR also makes a small but statistically significant improvement in the low-field Q 0 (see Fig. 26).It is interesting to note that this cannot be directly associated with field emission at these low fields, and the result remains somewhat surprising.One speculative possibility is that some fraction of the metallic conducting particulates assumed responsible for field emission at higher gradients are located on surface areas with high magnetic field, and can therefore influence the Q 0 at lower fields.A rough estimate suggests a small fraction of a square millimeter of normal conducting material would be sufficient to create the observed difference in the average Q 0 .
Finally, Fig. 27 shows the breakdown in limiting criterion of the usable gradient before and after retreatment (for those cavities having undergone retreatment).The reduction in field emission is clearly evident.
Seventeen cavities underwent HPR retreatment after they were found to be limited by "breakdown" between 10 and 27 MV=m with no or low field emission in the initial test.Most of these retreatments were done as a result of a NC of either the cavity or the test itself.As a result, the retreatment action included a partial or complete reassembly of the flanges before the final HPR treatment.The average maximum gradient and low-gradient Q 0 (measured at 4 MV=m) of the initial test was ð17.6 AE 5.1Þ MV=m and ð2.3 AE 0.4Þ × 10 10 , respectively.After the HPR retreatment the average maximum gradient was ð20 AE 7.8Þ MV=m.The increase of gradient and gradient rms spread was mainly caused by two cavities showing an improvement of 14 and 21 MV=m, respectively.The Q 0 value at 4 MV=m after retreatment was ð2.5 AE 0.3Þ × 10 10 .Neither result represents a significant difference.It can be concluded that a HPR retreatment applied on breakdown-limited cavities did not result in a significant improvement of the performance.since 3% of the cavities were accepted with lower gradients. 13 Figure 29 and Table VII show the usable gradient distributions and statistics, respectively, for the first and final (accepted) cold vertical tests (the latter corresponding to the usable gradient distribution in Fig. 28).The impact of selective retreatment of low-performance cavities is clearly evident, with the average usable gradient being increased by approximately 2 MV=m, and the yield at 20 MV=m being increased from 86% to 97%.The rms is also reduced from 7.2 to 5.1 MV=m.The breakdown of the limiting criterion for the final accepted usable gradients is given in Fig. 30, where comparison with Fig. 14 in Sec.IV B clearly shows that the overall field emission has been reduced.

E. Final performance
Finally, Fig. 31 shows the distribution of Q 0 as measured at 23.6 MV=m for the first and final (accepted) cold vertical test.As expected from the Q 0 criterion for the usable gradient definition (≥10 10 ), the low value tail below 10 10 is reduced but not completely removed; these cavities are predominantly those with usable gradients between 20 and 26 MV=m.

F. Performance comparison
In this section the gradient performance of the European XFEL cavities will be compared to available historical TESLA-type cavity data.
In preparation for the European XFEL cavity production, about 45 nine-cell cavities produced for the FLASH accelerator at DESY [47] have been surface treated, tested, and analyzed [48] following the two final surface treatment strategies adopted for the European XFEL series cavities [3].The mechanical production of the FLASH cavities took place at the same vendors EZ and RI (formerly Accel Instruments) as for the European XFEL cavity production but in partially different fabrication facilities.The main bulk EP (initial surface removal of 110 or 140 μm depending on the choice of final surface treatment, see Sec.II B) was performed at either Accel Instruments, DESY, or Henkel Lohnpoliertechnik. 14The final surface treatments took place exclusively at DESY.The vertical tests have been performed in the "Hall 3" test facility, which was not equipped with the second x-ray detector at the bottom of the cryostat as is the case for the vertical test setup in AMTF.Most of the tests have been performed without HOM feedthroughs in CW mode, compared to the "long pulse" mode used in AMTF.
In general the main conclusions described in [48] have been confirmed by the European XFEL cavity production: First, both surface treatment schemes final EP and flash BCP have proven successful for series cavity production.It is notable that the different schemes have been applied to cavities of both vendors in [48] (see Table VIII).
Second, all average gradients "as received" during European XFEL series production exceed the FLASH production by 2-4.5 MV=m, clearly demonstrating the successful knowledge transfer to industry and a stable well-established industrial production cycle.The lower average gradient for EZ cavities is caused by the recipe (flash BCP) and by some cavities showing an early quench (see Sec. VA).
Third, the application of HPR as an effective retreatment in the case of field emission (see Sec. IV D) could be convincingly confirmed.
Finally, in [48] no difference for the gradient yield "with He tank" and "without He tank" was demonstrated.Again this was confirmed by the European XFEL series  The gradient of these cavities could either not be improved or retreatment was not attempted due to schedule constraints.
production, but a difference in the quality factor at 4 MV=m was observed (see Sec. V F).
Another interesting comparison is with the maximumgradient yield data published in the technical design report for the International Linear Collider (ILC) project [49], which included 52 cavities tests from 2006 to 2012 performed at Cornell University, DESY, Fermilab National Accelerator Laboratory (FNAL), Thomas Jefferson National Accelerator Laboratory (TJNAL), and High Energy Accelerator Research Organisation (KEK).In this case we can directly compare the RI cavity production for European XFEL, as the ILC base line preparation recipe (first pass) is mainly identical to the European XFEL final EP treatment (adopted by RI). Figure 32 shows a clear improvement for "as received" European XFEL series production compared to the first-pass ILC data for the 28 MV=m maximum gradient yield, which can be interpreted as the benefit of an established industrial series cavity production.However, the yield at 35 MV=m maximum gradient remains essentially unchanged.(The concept of "usable gradient" was not formally defined for the ILC and therefore no direct comparison is possible.)

V. DETAILED CAVITY PERFORMANCE STUDIES
A. Vendor differences and impact of final surface treatment recipe In Sec.IVonly the total cavity production was considered and no distinction between the two vendors was made.In this section, first the differences of the cavity performance between the two vendors is analyzed, while in the second  part an analysis of the impact of the choice of final surface treatment (flash BCP versus final EP) is presented.

"As received" vendor dependence
In general there was a small but statistically significant difference in the average cavity performance between RI and EZ cavities.Figures 33 and 34 show the vendor performance comparison for "as received" maximum and usable gradient, respectively.
In both cases cavities manufactured by RI showed an average gradient approximately 10% higher than EZ, with the average maximum (usable) gradient for RI and EZ being 33.0 MV=m (29.0 MV=m) and 29.8 MV=m (26.4 MV=m), respectively.The dominant reason was assumed to be the application of final EP by RI as the last surface preparation step as opposed to the flash BCP used by EZ (Sec.II B), which is known to affect the socalled high-field Q slope typically seen in gradients above ∼25 MV=m (see Chap. 5 in [40]).This is discussed in more detail in the next section.The average EZ performance was also further slightly reduced compared to RI by the higher number of low performing cavities (below 20 MV=m), which is likely due to differing fabrication techniques between the two vendors.
Figure 35 shows the low-field Q 0 (measured at 4 MV=m) "as received" distributions for both vendors.The mean values are 2.1 × 10 10 and 2.2 × 10 10 for RI and EZ, respectively, a small but statistically significant higher value for EZ cavities by ∼5%.

Flash BCP versus final EP surface treatment
It is well known that BCP tends to produce a relatively large high-field Q slope compared to EP (Chap.5 in [40]).For the cavity production, the application of flash BCP by EZ as the final surface treatment manifests itself in a lower Q 0 performance at high gradients, effectively reducing their usable gradient (due to the Q 0 ≥ 10 10 acceptance threshold) as well as their maximum achievable gradient (due to the 200 W power limit in the cold vertical test).
In order to study this effect statistically for the production cavities in an unbiased fashion, a subset of cavities with no known "nonconformance" to the standard production procedure was used.Typically rejected cavities were those having clearly identified surface defects or some other nonconformance during fabrication.In addition, the final "accepted" test was taken (as opposed to the "as received" test, see Sec.IV E), in order to reduce the influence of field emission.Figure 36 plots for each vendor the recorded Q 0 value at the maximum achieved gradient in the vertical test.
The clustering of EZ cavities (orange points) at lower Q 0 values on the 200 W constant forward power (dashed gray) line is an indication of the stronger Q 0 slope (in particular when compared to the predominantly higher Q 0 at the lower gradients).By comparison, the RI data (blue points) do not show such a strong dependency and in general go to higher gradients before reaching the power limit.Note that data points above the dashed line are in general limited by quench.
In order to further quantify the difference in high-field Q slope between EZ and RI cavities, the following simple ad hoc figure of merit (s) was employed: where E max is the maximum field achieved during the power-rise measurement.Figure 37 shows the resulting distributions for RI and EZ cavities.The strong Q slope for flash BCP EZ cavities is clearly evident.The average value of s is ð− 4.1 AE 0.1Þ × 10 8 =MV m −1 and ð− 7.7 AE 0.2Þ × 10 8 =MV m −1 for RI and EZ cavities, respectively (mean AEσ= ffiffiffi n p ).Although Fig. 37 clearly indicates the larger high-field Q slope for EZ (flash BCP) cavities on average, it is important to note that there are some RI cavities (final EP) that show equally large Q slope, as can be seen by the long tail in the blue histogram.Equally there are examples of EZ cavities showing no or little high-field Q slope.More detailed analysis is merited requiring more sophisticated fits based on physical models to further quantify this behavior.

B. Impact of niobium material
The niobium material for the accelerating cells of the cavities was delivered by Tokyo Denkai, Ningxia OTIC, and SE Plansee.The final performance for 816 cavities accepted for string assembly was analyzed with respect to the material vendor (Fig. 38).There was no observed difference between cavities fabricated from Tokyo Denkai or Ningxia OTIC material (the dominant number of cavities).However, the 174 cavities manufactured from SE Plansee material showed a small but statistically significant higher average value for both the maximum gradient (E max ) and Q 0 ð4 MV=mÞ.As all delivered material batches from all three material vendors fulfilled the specifications, no obvious explanation can be given.The observed behavior will be the topic of a further detailed analysis.

C. Impact of infrastructure
In the AMTF hall two test cryostats (XATC 1=2) were available, both capable of taking one of six vertical test inserts each carrying at most four cavities (see Sec. III C 2). Figure 39 shows the top view of the test cryostats indicating FIG.36.Q 0 at the maximum achieved gradient for "standard" accepted cavities, separated by vendor.The gray dashed curve represents the 200 W power limit in the cold vertical test.the numbered locations of the cavities, which could not be rotated.
A statistical analysis was performed in order to exclude any systematic differences between the test infrastructure (test cryostat, insert number, and cavity location in the cryostat) on the cavity performance, notably the usable gradient (E usable ) and Q 0 ð4 MV=mÞ.
While no significant difference could be seen between the two test cryostat, studies of the six individual inserts were found to produce variations in both the E usable and Q 0 ð4 MV=mÞ.Figure 40 shows the example of the average Q 0 ð4 MV=mÞ for each of the test inserts.
Inserts 1 and 6 showed statistically significant lower and higher average values of Q 0 ð4 MV=mÞ, respectively, suggesting a slight bias for both by approximately AE6%.To date no possible cause for this difference has been identified.Analysis of the historical usage of the inserts over the three-year testing period showed no correlation with cavity performance trends.The observed insert dependency cannot be explained by individual cables at the inserts, since each cable is calibrated for each test.The observed fluctuations of the cable damping are understood, quantified, and well below the magnitude of the observed effect [41].One possible explanation of the Q 0 ð4 MV=mÞ effect is the detailed magnetic environment of each individual insert, which still has to be checked, especially under cryogenic conditions (see Sec. III C 3 for a description of room temperature magnetic measurements).
A similar study has been made with respect to the cavity position in the cryostat.Figure 41 shows the results for Q 0 ð4 MV=mÞ.In this case (and also in the usable gradient) a much clearer bias between positions 1 and 2 and positions 3 and 4 can be seen (approximately 8% difference).One possible explanation lies in the test stand geometry with respect to the rf connections.The rf input power circuit was first split in a so-called relay box housing three relays (two outputs per relay), distributing power to positions 1=2 and 3=4, after which the circuit was further split by two additional relays to each of the individual positions (see Fig. 39).Though this is the only known systematic difference with respect to positions 1=2 versus positions 3=4, it seems unlikely to be the reason for the observed bias since each insert had its own independent relay box placed on the top plate, requiring the cause to be systematic across all six inserts.Moreover, all typical rf checks and calibrations performed for the cold vertical tests showed no suspicious behavior; nevertheless a detailed hardware study of the systematics will be made in the near future.In addition, the above mentioned additional checks of the magnetic environment will naturally include a possible dependence on the cavity position.
The typical observed maximum spread in the average values is ∼1.5 MV=m and ∼0.4 × 10 10 for E usable and Q 0 ð4 MV=mÞ, respectively.Practically speaking, such small systematic shifts in the results are not particularly relevant for the overall cavity production.Nonetheless, for the statistically significant outliers discussed above, it is important for future measurements to identify the cause of the apparent systematic bias associated with the test infrastructure, and further hardware tests and measurements are planned.

D. High-pressure rinse analysis
A crucial part of cavity preparation was the HPR, where ultrapure water with a pressure of 95 to 105 bar was used to clean the inner surface of the cavities.This step was applied several times and with different durations during the European XFEL cavity fabrication according to the recipes for both surface preparation methods as described in Sec.II B. The success of recovering low-performance cavities by an additional HPR only (see Sec. IV D) raised the question if whether or not more HPR cycles during production would systematically improve the cavity performance.
The rf performance values E max , E usable , and Q 0 ð4 MV=mÞ were statistically analyzed based on which of the two HPR stands were used at each of the vendors (infrastructure impact), and then by the number of HPR cycles applied (i.e.duration of HPR) at both DESY and the vendors.Variations of the duration of HPR were caused by nonconformities during the process of cavity surface preparation (e.g.failed leak checks followed by reassembly and re-HPR; nonconformities during chemical treatments; and nonconformities during an HPR cycle itself) or the incoming inspection (see Sec. III C 1).
For all 831 cavities, no correlation with performance could be identified with respect to the HPR stand or the number of cycles.Restricting the analysis to higherperformance cavities with no field emission likewise showed no correlation.We therefore conclude that variation in application of HPR beyond the standard specified procedure had no influence on the performance of the cavities.Together with the success of HPR as a highly effective retreatment process, especially for field emission limited cavities, we conclude that the source of field emission lies in the handling steps following the final HPR cycle (i.e. assembly of the beam tube flange, vacuum operations, or transport; see Sec.V F).Furthermore, the analysis indicates that the specified standard number and duration of HPR cycles during the surface preparation is adequate.E. Degradation and processing "Processing" and "turn-on" of field emission are wellknown effects during vertical testing (see for example Chap.12.2 in [40]).Turn-on of field emission results in a decrease of the Q value, the maximum gradient, or both, together with an enhanced radiation level.If the resulting field emission cannot be "processed" away by applying higher rf power, it will lead to a permanent reduction ("degradation") of the cavity performance.
For all vertical acceptance tests no administrative gradient limitation was applied up to the maximum rf input power of 200 W (see Sec. III C 2).Very few tests with strong radiation were limited by the radiation protection system installed outside of the cryostats.Moreover, no dedicated processing procedures like helium processing or high peak power processing (see Chaps.13.5.1 and 13.7 in [40], respectively) were applied.During cold vertical tests, the power rise measurements [Q 0 ðEÞ curves] were repeated until two stable identical curves were obtained.Hence, the comparison of first and last power rise provides a handle to judge the behavior of the cavity during testing.The change in gradient (Δ) in the following analysis was defined as the usable gradient of last power rise minus the usable gradient of first power rise.Thus a positive value represents processing during the test cycle, while a negative delta indicates a so-called degradation.In the following, only values of jΔj ≥ 10% are considered relevant degradation or processing.
Power rises which were limited due to technical reasons and not the cavity itself were excluded from the analysis.Furthermore, a few pathological cavities showing low gradient behavior were also excluded.This resulted in a data set of 1100 usable vertical tests for use in the statistical analysis.An overview of the observed relative Δ versus the usable gradient measured in the first power rise can be seen in Fig. 42.
Of the vertical tests shown in Fig. 42, 89 (8.1%) showed a degradation, while 88 (8.0%) showed a larger usable gradient in the last as compared to the first power rise.
Table IX gives a summary of all vertical tests and the corresponding numbers for degradation and processing as defined above.The first data column shows the results for all tests, while the second and third columns distinguish between cavities that achieved either below or above 31 MV=m (the operational limit in the linac and in the AMTF cryomodule test stand [50]) in the first power risea likely choice of administrative limit in the cold vertical test should one have been applied.While there is a small loss of gradient for cavities going above 31 MV=m, due to a few cases of degradation, this can be practically ignored.It should also be noted that if the tests had been administratively limited at 31 MV=m, then the knowledge of the true performance limits of over one-third of the cavities would not have been known, and the average performance reported in Sec.IV would have been significantly less.The choice not to impose an administrative limit in the vertical test thus appears to be justified.

Cavity performance-production related issues
Beyond the chemical surface preparation, the cavity production process included a number of additional procedures which could have possibly influenced the cavity performance.Specifically, four key procedures were identified for further study: titanium 2-phase pipe repair [51], cavity transport, mechanical surface repairs (grinding), and helium tank welding.These four studies are briefly reported below.Impact of titanium 2-phase pipe repair.-Thelongitudinal welded titanium 2-phase pipe is a part of the He tank and has to fulfill the requirement of the pressure equipment directive as well as internal DESY standards.As described in detail in [51], pores discovered in the weld were out of tolerance and more than 750 cavities had to be repaired, including some fraction that had already received a cold vertical test.In most cases the end parts of the 2-phase pipe were cut and replaced.It is worth noting that the repair was done with all flanges sealed and the cavity under vacuum.
To check the possible impact of the repair on cavity performance, the modification was first made on one cavity whose performance was then remeasured.No degradation was observed and the repair process was then applied to the remaining cavities.A retest of all repaired cavities was not possible due to schedule constraints, but a random selection of the repaired cavities was tested to provide some element of quality control.To make an accurate statement, only cavities having had the 2-phase pipe repair and no other kind of surface treatment in between the tests were taken into account (a subset yielding a total of 20 cavities).Table X gives the statistics for E max , E usable , and Q 0 ð4 MV=mÞ.The deviations in the averaged gradients between the measurements before and after repair are less than 2% and are well within the range of the accuracy of the measurement.An increase in field emission was also not observed.Hence based on these sampled statistics it is assumed that there was no significant impact on any of the cavities which underwent the repair.Impact of horizontal transport under vacuum.-Thedistributed sites for cavity surface treatment, vertical acceptance test, and cryomodule string assembly made the reliable and cost-effective transport of the cavities "ready for test" (see Sec. II B) an important part of cavity production.Several investigations during the preparatory phase were made to find a suitable shock absorbing system for the horizontal cavity transport in boxes [17].In this section, the impact of the long-distance cavity transports (vendors to DESY and DESY to CEA Saclay) by truck is discussed.A total of 20 cavities with vertical tests before and after transport were identified (as above without any additional surface treatment): 12 cavities were transported from DESY to and back from the module assembly site (CEA Saclay); and 8 cavities were sent between DESY and EZ.The results are shown in Table XI.The deviations in the averaged gradients-before and after shipment-are less  [3].Most of these nonconformities were removed by a grinding procedure as described in Sec.II B. Both E max and E usable are approximately 3 MV=m higher than for all "as received" EZ cavities (29 MV=m and 26.4 MV=m, respectively).The quality factor Q 0 ð4 MV=mÞ is slightly increased compared to all "as received" EZ cavities.
In the cases where the surface defect was identified using OBACHT after a low-performance vertical test, both E max and E usable increased significantly by 9 MV=m and 11 MV=m, respectively, indicating the effectiveness of the repair.The results after grinding were in good agreement with the average maximum and usable gradients for all "as received" EZ cavities.The Q 0 ð4 MV=mÞ was unchanged within the standard error, and is also slightly higher than the overall average value of 2.1 × 10 10 .
As a remark it should be mentioned that in a few cases applying the BCP retreatment process (see Sec. II B) instead of grinding was sufficient to remove the surface defects.On the other hand several cavities with defects on the inner cavity surface were not treated since they fulfilled the acceptance criteria (see Sec. III B).Unfortunately a small number of cavities with a specific type of surface defect, called erosion, could not be repaired.These defects, typically on the welding seam, were so deep that a repair by grinding or BCP was not attempted.
In conclusion, the above results indicate that mechanical grinding was an extremely effective measure in recovering poor performing cavities with identified mechanical defects, and that the results of grinding produced cavities with the same average performance as nongrinded cavities.
Impact of He tank assembly.-TheHe tank welding process was developed during the preparatory phase of the European XFEL cavity production, and included a cleanroom compatible field profile measurement system (FMS) [52,53].At that time and with limited statistics no difference in the cavity performance "with" and "without He tank" was observed (see Sec. IV F).Based on a beadpull measurement the FMS allows the frequency and field-profile monitoring during the various steps of the He tank welding process while keeping the cavity closed and under clean conditions.Nevertheless several additional handling and cleaning steps are necessary, which hold the potential risk of a contamination and hence performance degradation.
Two different analyses were made: first, a direct comparison "before" versus "after" He tank assembly; second, a statistical comparison of the 23 HiGrade cavities without He tank "as received" (see Sec. II A) versus the 741 "as received" series cavities.The cavity performance results are shown in Table XIV.As noted above, a total of 23 cavities had two vertical tests with a standard tank assembly procedure in between and no surface treatment other than the HPR cycle applied after the tank welding procedure (i.e. after the removal of the FMS).None of the rf parameters E max , E usable , or Q 0 ð4 MV=mÞ showed any statistically significant change.Noteworthy is however the higher Q value in general for this subset of cavities (∼2.7 × 10 10 ), as compared to the overall average (2.14 × 10 10 ).This may be partially attributed to the fact that the sample includes eight HiGrade cavities subsequently equipped with a He tank as well as seven cavities that had already undergone retreatment.To further confirm the benign effect of tank welding observed from the direct before and after analysis, the "as received" performance of 23 ILC HiGrade cavities (without tank) and the series cavities (741 with tank) were statistically compared.While the average E max is unchanged, the observed average difference in Q 0 ð4 MV=mÞ is ∼0.2 × 10 10 , which would indicate a weak negative influence of the He tank assembly on the low-gradient Q value.In addition, statistically weak differences between the vendors have been observed.Despite these small effects, we conclude that the He tank assembly procedure had little or no influence on the cavity performance.

Estimate of residual surface resistance
For a better production characterization, the Q 0 ðEÞ performance of 24 cavities was measured at 1.8 K in addition to the standard 2 K measurements made during the cold vertical test.With only two measurement points, the deconvolution of the surface resistance into BCS (R BCS ) and residual (R res ) parts (see Chap. 3.1 in [30]) is not possible.Assuming a low BCS surface resistance of ð8-10Þ nΩ at 2 K for a cavity after a 120 °C bake (confirmed by earlier measurements) and a residual resistance of about 4 nΩ [equivalent to R BCS (1.8 K)], a ratio of the observed quality factors of about 1.5 is expected.This is in good agreement with the observed ratio of 1.4 AE 0.2 found on the 24 analyzed cavities (Fig. 43).Furthermore the above assumptions fit well to the observed average "as received" Q 0 ð4 MV=mÞ of 2.1 × 10 10 , corresponding to a surface resistance of ∼13 nΩ.Together with five cavities (infrastructure commissioning and HiGrade) where the residual resistance could be fitted to ð5.1 AE 0.7Þ nΩ from the temperature dependent surface resistance measurement, an average residual resistance of ð4-6Þ nΩ has been estimated.

Analysis of cool-down procedure
It has been reported in [54,55], that the cool-down dynamics across the critical temperature T c has a significant influence on the observed Q 0 , which is most likely due to trapped flux [56].We investigated if such a correlation could be observed during the European XFEL cavity production.
As mentioned in Sec.III C 4 only one insert was equipped with Cernox temperature sensors which allowed a direct measurement of the helium temperature profile at the cavity positions.This instrumentation was only used for commissioning.For regular operations, TVO temperature sensors were glued to the outside of the cryostat.This instrumentation was designed specifically for operation and control of the cryogenic system only, and cold vertical tests were performed in stable cryogenic conditions.The analysis of the available data showed that a time-resolved cooldown rate across the cavities could not be deduced accurately enough to make any statements about the influence of cool down [57].
In passing we note that hydrogen Q disease was not observed in any of the cold vertical tests, within the given measurement uncertainties.

VI. SUMMARY
The production of over 800 cavities for the European XFEL by industry and the associated cold vertical testing at DESY has been a remarkable success.The total production was divided equally between E. Zanon Spa.(EZ), Italy, and Research Instruments GmbH (RI), Germany.The cavities sent for cryomodule assembly at CEA Saclay achieved an average maximum gradient of approximately 33 MV=m, reducing to ∼30 MV=m when the operational specifications on quality factor (Q 0 ) and field emission were included (the so-called usable gradient).Only 16% of the cavities required an additional surface retreatment to recover their low performance (usable gradient less than 20 MV=m).These cavities were predominantly limited by excessive field emission for which a simple high pressure water rinse was sufficient.Approximately 16% of the cavities also received an additional HPR, e.g.due to vacuum problems before or during the tests or other reasons, but these were not directly related to gradient performance.The successful achievement of the overall high performance and unprecedented production rate (a total of eight cavities per week) was due in part to a careful transfer to both vendors of the decades of knowledge at both DESY and INFN Milano.Both vendors needed to expand existing infrastructure as well as develop new installations to meet the stringent demands of the cavity fabrication and chemical surface treatment at the required delivery rates.Careful automated workflow, extensive QC, and documentation was an essential part of the build to print process, including the numerous inspections and measurements.Furthermore, the process would not have been successful without the close collaboration of industry and the lab personnel, which continued throughout the entire production period.
The team from IFJ-PAN Krakow successfully performed over 1300 cold vertical tests of the cavities (including repeated tests) at the purpose built Accelerator Module Test Facility at DESY.Average testing rates exceeded 10 cavities per week, peaking at up to 15 cavities per week.Heavy automation of the test procedures was essential to achieve the test rates as well as maintaining consistency of the tests themselves and the associated documentation and data storage.
The in-depth statistical analyses presented in this report have revealed several features of the series produced cavities.The overall production quality from the vendors increased during the second half of production.The application of an additional HPR proved extremely effective for the low-gradient performing cavities (mostly fieldemission limited), recovering nearly 80% of these cavities to gradients higher than 20 MV=m.Although both vendors exceeded the specifications, cavities produced by RI performed 10% better on average in gradient than those from EZ, which has been demonstrated to be predominantly due to the fundamental difference in high-field Q slope between the final surface treatments applied (final EP and flash BCP, respectively).In general, the average performance of the European XFEL cavities has slightly improved on the historical TESLA-type cavity performance (FLASH cavities and ILC) taken over the last decade.
Despite the success, there remain questions and "lessons learned" for future projects.Although the average performance is impressive, the large spread in the observed results (ranging from 10 to 40 MV=m) is still a strong indication that the production could be improved.The large spread (in particular the low-gradient tail) coupled with the high average was the driving reason why the acceptance threshold for retreatment was reduced from 26 to 20 MV=m early in the production.Understanding the relevant factors affecting the performance of individual cavities which essentially underwent the same process will be an important part of future studies using the large legacy of data from the series production.Furthermore, systematic studies of the difference of the test infrastructure have revealed small (<10%) but statistically significant bias in the cavity performance results.These differences, together with a better analysis of the overall uncertainties and measurement errors, require further study.
Finally the European XFEL experience is already proving to be of great benefit to future projects such as Linac Coherent Light Source II (LCLSII) [58], currently under construction at SLAC, California, USA, and the European Spallation Source (ESS) [59], under construction at Lund, Sweden.Looking further into the possible future, the European XFEL has laid a strong foundation for the possible construction of the ILC [49] in Japan.Until such time that the ILC is built, the European XFEL will likely remain the largest deployment of the TESLA technology for some time to come.

APPENDIX: DATABASE ANALYSIS FLAGS AND THEIR DEFINITIONS
Table XV summarizes the flags used to categorize the vertical tests in the database.For the retreatment secondary categories, NCout and NCin refer to external or internal nonconformities respectively.
Table XVI provides a mapping from the detailed flags to the summary flags used in the analysis presented in the main body of the report.

FIG. 1 .
FIG. 1. 3D model of the series XFEL cavity equipped for delivery to DESY.

FIG. 3 .
FIG.3.Workflow of the vertical acceptance test.A simplified list of "decisions" (applied after each vertical test) is given in the gray box and their related branch points are indicated by the red numbered circles.

FIG. 4 .
FIG. 4. Section diagram of the vertical test cryostat.Note that only three of the four cavities are visible.

FIG. 5 .
FIG. 5. Left: vertical test inserts ready for test.Right: 3D view drawing of a test insert supporting four equipped cavities.

FIG. 6 .
FIG. 6. Magnetic field distribution inside the cryostats XATC1 (top) and XATC2 (bottom) of the bottom 2-m vertical section of the cryostat.The vertical extent of the cavities is indicated in gray.

FIG. 10 .
FIG.10.Reasons for cavities requiring a second cold vertical test.Note that the percent figures indicate the fraction of the total number of cavities (831) and not the fraction of the chart itself (corresponding to the 28% of cavities with more than one test in Fig.9).

FIG. 17 .
FIG. 17. Q 0 distributions for the "as received" cold vertical tests, measured at 4 MV=m (generally field emission free) and 23.6 MV=m.(Note that 15% of the cavities did not reach 23.6 MV=m in the "as received" cold vertical test and therefore do not appear in the respective histogram.) FIG. 19."As received" usable gradient performance by delivery month.The box-whisker symbol represents the gradient distribution for the respective calendar month.The dark blue joined dots indicate the monthly average gradient.The number of cavities delivered in each month is given at the base of the chart.The red dashed horizontal lines indicate the average performance over the first and second halves of the production, indicating improved average performance during the latter.(The last bin contains all remaining cavities arriving at DESY after 1 November 2015.)

FIG. 20 .
FIG.20.The fractional numbers of cavities having "as received" usable gradient in the indicated ranges for quarterly production.A clear improvement (reduction) of the number of cavities with gradients less than 20 MV=m can be seen in the second half of production.
FIG. 22. Breakdown of the reasons for the first cavity retreatment at DESY.Field emission, low Q, and quench are performance related (total of 66%) while leak and other are nonperformance driven.(Total chart represents 16% of the cavities.)

Figure 28 and
Figure 28 and TableVIshow the final (accepted) gradient distributions and statistics, respectively, for cavities used for string assembly, which includes those cavities having undergone one or more retreatments.The yield at 20 MV=m is not 100% as might be expected,

FIG. 33 .
FIG.33.Distribution and yield of the "as received" maximum gradient separated by vendor.

FIG. 37 .
FIG. 37. The distribution of the high-gradient (≥25 MV=m) linear Q slope expressed by the figure of merit (s) of the measured Q 0 ðEÞ curves, separated by vendor.

FIG. 39 .FIG. 41 .
FIG. 39.Schematic top view of both test cryostats (XATC1=2)showing the cavity positions inside the stands and the rf input power distribution.The orientation of the insert in the cryostat was always the same; it cannot be rotated.The insert structure is not visible.

FIG. 42 .
FIG. 42.Relative change of usable gradient (Δ) with respect to the usable gradient as determined from the first power rise.Vertical tests with degradation or processing (jΔj > 10%) are shown outside the horizontal dashed lines.Note the 1 MV=m binning of the usable gradient causes overlaying data points.More than half of the vertical tests (573 out of 1101) showed no difference in the usable gradient within the vertical test (represented by the many overlaying dots at Δ ¼ 0).

TABLE I .
Total number of cavities produced by vendors.

TABLE II .
Statistics of cavity returns (before and after the first cold vertical test).The total of 142 returns corresponds to 129 individual cavities (13 cavities were returned to the vendor twice).

TABLE III .
Number of vacuum NC during vertical acceptance test.

TABLE IV .
Total number of cavities used in the analysis (see Sec. II for more details).The total number of 831 differs from the number given in Sec.II A (832) due to the excluded HiGrade cavity.
FIG. 8. Stacked bar chart showing the number of cold vertical tests by calendar month for RI (orange) and EZ (blue) cavities based on test date.The dashed line is the total average test rate taken from 1 October 2013 to 31 November 2015, which is considered the peak testing period (after ramp up and before ramp down).

TABLE V .
Key statistics for "as received" cold vertical tests (cavities).Breakdown of the categories for the first cold vertical test in the database of the 831 cavities.

TABLE VI .
Key statistics for accepted cavities (including retreatments).Comparison of the usable gradient distributions for the first and final (accepted) test.

TABLE VII .
Comparison of the key statistics for the first and final (accepted) tests.

TABLE VIII .
Comparison of the maximum and usable gradients (average AErms in MV=m) for the European XFEL series production and FLASH cavities.FIG.32.Comparison of yield data for the first-pass maximum gradient taken from the ILC technical design report, and the European XFEL "as received" final EP treatment.The European XFEL data points were manually added to the original figure taken from [49] using the same error calculation formulas.Numbers in parentheses refer to cavity sample size.

TABLE IX .
Overview of degradation and processing statistics for all measured power-rise data (total), as well as those cavities achieving below or above 31 MV=m on the first power rise.The average Δ values were calculated over the corresponding number of tests.(Reported error is σ MV=mÞ 2.23 × 10 10 (AE0.3 × 10 10 ; AE0.07 × 10 10 ) 2.34 × 10 10 (AE0.3 × 10 10 ; AE0.08 × 10 10 ) than 1%.An increase in the field emission was also not observed.Based on these sampled results it was assumed that transport had no influence on the average performance of all series cavities.Impact of grinding of surface defects.-Innersurface defects were identified during cavity fabrication at the vendors as well as after cold vertical tests.Repair by local grinding took place at EZ [20].The quantities of affected cavities are given in Table XII.The cavity performance results are given in Table XIII.During cavity production different types of surface defects were identified.A detailed overview is given in

TABLE XII .
Cavities requiring repair of identified surface defects.

TABLE XV .
Cold vertical test analysis flags.issues with the rf system or results.NC at Saclay Test performed after a cavity was returned from the cryomodule assembly plant at CEA Saclay, due to a nonconformity.warm-up cycle Retests related to specific cool-down R&D.other Miscellaneous (generally minor) reasons for not falling into one of the above categories.Retreatment quench Cavity was sent for retreatment due to low breakdown gradient.low Q Cavity was sent for retreatment due to low Q 0 performance.FE Cavity was sent for retreatment due to excessive field emission.NCout fibers Cavity underwent retreatment as a result of the observation of fibers (incoming inspection).NCout mechanical Cavity underwent retreatment due to the observation of external mechanical defects (incoming inspection).NCout 3DT Cavity underwent retreatment as a result of nonconformity with the mechanical axis measurements (incoming inspection).NCout vacuum Cavity underwent retreatment as a result of vacuum nonconformity.NCin defect Cavity underwent retreatment as a result of an identified internal surface irregularity (optical inspection).NCin defect, grinded Cavity underwent retreatment as a result of an identified internal surface irregularity which underwent local grinding (optical inspection).NC at Saclay Cavity underwent retreatment as a result of a nonconformity at the cryomodule assembly plant at CEA. other Miscellaneous (generally minor) reasons for not falling into one of the above categories.

TABLE XVI .
Summary flags used during analysis.