Universal picosecond timing system for the Facility for Antiproton and Ion Research

The present article presents a system making it possible to produce time information at spatially separate points with picosecond range precision. By using this time information, the accelerator and storage rings of the Facility for Antiproton and Ion Research (FAIR) are to be synchronized. To do this, the rf cavities of these ring accelerators have to be controlled by signals having different phases and frequencies (0.4 to 5.4 MHz). Some frequencies of the signals are variable during acceleration due to what is referred to as ramp control. To enable synchronization of all these different signals, also during acceleration, the signals for controlling the cavities are not sent directly by the timing system, but instead two clock signals of constant frequency are sent. These clock signals are produced phase synchronously at different points up to 1 km apart and represent the time information. With the help of these clock signals, it is then possible to synchronize frequency generators that produce the signals actually needed for the cavities. Because of the universal character of the time information produced, it can be used not only to control the cavities but also to synchronize other processes. To transmit the clock signals, an optical network with dense wavelength division multiplex methods is used. The delay of the clock signals is measured and with the help of the delay information a reference generator produces, at the end of each transmission line, a phase-synchronous and phase-stable time reference. Since the delays of the clock signals are not constant due to environmental influences, they must be determined regularly. Using a prototype of this system, a precision of the time information of 21.2 ps on average was achieved. The short-term jitter exhibits a standard deviation of 7.57 ps. In addition to the description of the system, detailed information on noise characteristics is provided that can also be used to optimize other optical systems for transmission of time information.


I. INTRODUCTION
The system presented here was developed to synchronize the electrical fields of the cavities in the future particle accelerators of the Facility for Antiproton and Ion Research (FAIR) (Fig. 1).Not only the cavities of a ring but also the rings amongst themselves have to be synchronized.The latter is necessary in order to transfer the bunches from one ring into another ring [1].This process is referred to as bunch-to-bucket transfer (Fig. 2).
The peculiar feature of synchronization in FAIR is that the cavities are to be operated at different frequencies (0.4 to 5.4 MHz) and that frequency ramps of 0.85 to 5.4 MHz have to be realized during acceleration due to the large mass of ions.In the systems known from the literature [2][3][4][5][6][7][8][9][10], generally a phase-stable reference signal is transmitted to the cavities from which a control signal of constant frequency is derived.These systems are not designed to produce phase-coordinated signals of rapidly changing frequency at different points.To make this possible at FAIR, it is planned to have signal generators located near the cavities to define their frequencies and phases.
To synchronize the cavities, it is thus necessary to synchronize the signal generators.The signal generators work according to the direct digital synthesis (DDS) principle.PHYSICAL REVIEW SPECIAL TOPICS -ACCELERATORS AND BEAMS 12, 042801 (2009) 1098-4402=09=12(4)=042801 (10) 042801-1 Ó 2009 The American Physical Society DDS is a method by which a sinusoidal oscillation can be produced digitally whose frequency is derived from a constant clock frequency [11], page 21.To enable synchronous operation of the signal generators, these must be fed phase-synchronous reference signals.Figure 3 shows the principle of synchronization of the signal generators and thus of the electrical fields of the cavities.
The target frequencies and phases are changed by means of control commands.For these control commands to be executed synchronously, two clock signals are needed.These clock signals are referred to as reference signal 1 and reference signal 2. They have the frequencies 50 MHz and 97.7 kHz.Reference signal 1 is used by the signal generator for digital signal synthesis, and reference signal 2 is needed to clock the command transmission and execution.The commands, received within a clock period of reference signal 2, can be executed in the following clock period at a predefined clock pulse edge of reference signal 1.As a result, the transmission of the command data is not time critical.This method, referred to as resynchronization, is shown in Fig. 4.
The purpose of the system presented here is to produce phase-synchronous and phase-stable reference signals at 13 spatially separate points of the facility.The phase shift between two reference points is not to exceed 514 ps, corresponding to 1 of the highest frequency occurring in the cavities.The system's accuracy is determined by reference signal 1.

II. BASIC PRINCIPLE
To generate the reference signals, two clock signals (200 MHz and 97.7 kHz) are transmitted from a central point to the reference generators (Fig. 5).These are able to derive from the clock signals, at several locations, reference signals which are frequency synchronous with one another but which exhibit a phase displacement Á' that depends on the respective delay of the clock signals n .To determine Á', the delays are measured.With the help of this information, phase corrections are effected in the reference generators and in this way the phases of the reference signals ' ref are synchronized.Since the delays are time variable due to environmental influences, they must be measured on a permanent basis.By using only one transmitting and one measuring unit instead of one for each branch, respectively, the accuracy of the system is increased whilst lowering its costs.
The frequencies of the clocks and the reference signals derived from them are listed in Table I.

III. SYSTEM DESIGN
To achieve the primary objective of phase synchronization, two new approaches are pursued.First, the dense wavelength division multiplex (DWDM) method is used for signal transmission and delay measurement, and second, phase correction is effected by means of reference generators operating according to the DDS (direct digital synthesis) principle.
Using the DWDM method it is possible to transmit signals on various optical carriers of different wavelengths over a common fiber.In this way the two clock signals and the measurement signal can be sent independent of one another.The decoupling of the measurement signal from the clock signals to be transmitted allows for the delays in the transmission fibers to be measured more precisely and constitutes a significant distinguishing feature versus already known systems such as [2][3][4][5][6]12].
The advantage of using a reference generator operating according to the DDS principle is that phase shifts of any size are possible with a very high resolution.This in turn means that any size of delay changes can be compensated in the transmission fibers.

Fiber
To save costs, no phase stabilized optical fibers [5][6][7][8][9][12][13][14] were used, but standard single mode fibers (SMFs).The phases of the reference signals are equalized and kept stable using reference generators.It is assumed that, by using fibers found in robust loose tube cables, only the change in temperature contributes to the change in the signal delay and mechanical stress does not have any effect.Because of the underground laying of the cables, the signal delay will merely depend on the average outside temperature (0 to 25 C) and the speed of signal delay change is low, less than 0.006 ps on 1 km fiber per second [12].Since the measurement system needs about 1 s to measure the signal delay of a transmission branch and thus correction data are available for each branch every 13 s, the system is able to detect all changes fast enough and in this way to compensate delay changes.

Transmission branch
The two clocks are optically multiplexed, i.e. they are modulated to two different optical wavelengths 1 and 2 and combined into one fiber in a multiplexer (Fig. 6).The two optical signals then pass through an add/drop multiplexer, the transmission line (SMF 1 km) and a fiber Bragg grating (FBG).In the demultiplexer, the wavelengths are separated again and fed to two separate receiver units.
The measurement signal for determining the delay is modulated to a third optical carrier M .Via a circulator, the measurement signal is delivered to the add/drop multiplexer which combines the signals 1 , 2 , and M .After passing through the SMF, all signals meet the FBG, which represents a wavelength-selective reflector that exclusively reflects M while letting through the other two signals.The measurement signal now returns, is decoupled in the add/ drop multiplexer, and fed to the measurement receiver via the circulator.

Star-shaped distribution
Figure 7 shows how the distribution to several points takes place.The optical signals are fed to several transmission lines via a power splitter.The attenuation of the splitter is compensated by an erbium-doped fiber amplifier (EDFA) [7].
With this approach, phase displacements in the optical transmitters, as described in [6,7,10,15], no longer matter because they affect all transmission branches equally and thus do not have any influence on the synchronization of the phases of the reference signals.

B. Measurement unit
Only one measuring unit is used to determine the signal delay (Fig. 7).By means of an optical switch, the measurement signals are switched sequentially to the different branches and a reflector for calibration.Any remaining systematic errors are, up to the FBG, of the same magnitude for all measurements and therefore irrelevant for the phase synchronization of the reference signals.For measuring, one sinusoidal oscillation f M after the other is modulated to M and a phase comparison of the outgoing and returning signal is performed (Fig. 6).This is done 5 times by a vector network analyzer with frequencies between 30 kHz and 6 GHz.From the phase values obtained, the signal delay of the clocks, taking account of dispersion, can be finally determined. 1 The accuracy of the measurement accu depends on the highest measurement frequency f max ¼ 6 GHz and the accuracy of the phase meter ' accu < 0:4 accu < 1 f max ' accu 2 Á 360 ¼ 92:6 fs: (1)

C. Reference generator
In Fig. 8, DDS units 1 and 2 are provided with the clocks and generate the reference signals with a phase displacement Á' which depends on delay .Since the delay is known, the Á' and thus the correction data ' cor can be calculated from it for the DDS units.Next, the DDS units perform a phase correction.This is carried out regularly in each system branch depending on the current delays.In this way the phases of the reference signals are synchronized.The resolution of phase adjustability ' res depends of the length of the phase/offset word n POW of the DDS unit (14 bit for the prototype) and the generated frequency.For reference signal 1, this is IV. PROTOTYPE All essential elements of the system were implemented in a prototype.In Fig. 9 the optical network and the measurement unit can be seen.All three optical transmitters each consist of a DWDM laser and an external modulator by which the optical carrier is modulated in its amplitude.The DWDM laser modules are all located in a mainframe.Their optical wavelengths were selected according to the recommendations of the ITU (International Telecommunications Union) and are listed in Table II  Modulation of the optical carriers takes place using Mach-Zehnder modulators located in the setup beneath the lasers.They have a bandwidth of 10 GHz.

V. PERFORMANCE
Noise added to the measurement signal and the clock signals during transmissions causes phase fluctuations (jitter).This jitter leads to time fluctuations in the generation of the reference signals and can cause imprecisions in delay measurements.For this reason, they must be kept as small as possible.No phase noise has been observed in the prototype.Therefore, only amplitude noise will be considered.

A. Noise considerations
During transmission of the signals, three relevant types of noise occur both in the transmission channels and in the measurement channel: amplitude fluctuations of the optical carrier RIN (relative intensity noise), shot noise, and receiver noise.It is assumed that all noise processes are approximately white and of Gaussian distribution.For comparison of the noise types, we introduce a relative noise-power density which, like RIN [16], page 272, refers to the DC power of the electrical signal delivered by the photodiode. 2 The value of RIN, unlike the other relative noise-power densities, is independent of the optical power fed to the photodiode in the receiver.
The relative power density of shot noise shot is identical to the ratio of the shot noise density (A 2 =Hz) [17], page 786, to the square of the mean value of the current [17], page 785, delivered by the photodiode since both variables are proportional to the absolute shot noise power density shot;abs and the DC power of the electrical signal P DC .For the relative power density of the shot noise it can thus be formulated: in which q stands for elementary charge unit, R for the sensitivity of the photodiode (responsivity), and P PD for the optical power fed to the photodiode in the receiver.
The relative power density of the receiver noise can be determined in a similar way.For this, the absolute noise density of the receiver (A 2 =Hz) must be put in relation to I 2 PD in Eq. ( 4) r is an auxiliary variable used to characterize optical receivers.It represents a virtual optical noise power density at the receiver.
In Fig. 10 all relative noise power densities and their sum are shown as a function of the optical input power at the receiver of the prototype.In the calculation with Eqs. ( 5) and (7), the following values were assumed: RIN ¼ À145 dB=Hz, 3q ¼ 1:6 Â 10 À19 As, R ¼ 1 A=W, and r ¼ 20 pW= ffiffiffiffiffiffi Hz p .It can be seen that with rising input power the sum of the relative noise power densities initially decreases linearly and is defined above À5 dBm almost exclusively by the amplitude fluctuations of the optical carrier RIN.Since the receivers used are able to convert optical signals up to an input power of about 0 dBm distortionlessly into electrical signals, the system was designed in such a way that all receivers are supplied with 0 dBm of optical power to achieve noise values that are as low as possible.This is feasible for two reasons: first, the attenuation of the DWDM components (multiplexer, etc., Fig. 6) with <1 dB is very low.Overall, the attenuation between the transmitter and the receiver in both the measurement channel and the transmission channel is less than 4 dB.Second, the attenuation caused by the splitter is compensated by means of the EDFA (Fig. 7).

Transmission channel
When amplifying the clock signals, however, the EDFA also causes an optical noise as a result of amplified spontaneous emission (ASE) [17], page 874 et seq.Its optical noise power density at the exit of the EDFA (Fig. 11) can FIG.10. (Color) Noise in all channels.RIN dominates above À5 dBm of optical input power compared to the shot and receiver noise.be calculated, using PLANCK's constant h, the frequency of the optical carrier , the noise factor F, and the amplification G.This optical noise, attenuated by the factor after transmission, meets the photodiode and causes various types of noise [17], page 873 et seq., all of which can be treated as negligible except for the signal-spontaneous noise. 4The density caused by the signal-spontaneous noise (A 2 =Hz) can be calculated using [17], page 874, 5 To obtain the relative noise power density s-sp (1=Hz), Eq. ( 9) is placed in relation to Eq. ( 4): Here, Eq. ( 8) and are used, in which G and P in stand for the amplification and input power of the EDFA (Fig. 11).By this, is obtained for the relative power density of the signalspontaneous noise.If G ) 1, then F > 2 [18], page 100, and for Eq. ( 12) it can be formulated by way of approximation s-sp % hF P in : Since the parameters h, , and F 6 are constant, the noise caused by the EDFA s-sp according to Eq. ( 13) is approximately only dependent on the optical input power and not on the amplification.In order to check this, the EDFA was fed two optical signals 1 and 2 (in each case P in ¼ 3 dBm ) 6 dBm EDFA input power).In this way the EDFA used, given the relatively high input power, is operated in saturation and delivers its maximum output power of 21.3 dBm ( ) P out ¼ 18:3 dBm per channel).The amplification is thus 15.3 dB and suffices to compensate for the losses of the required 1 Â 16 splitter of approximately 15 dB.This results, according to Eq. ( 12), with the parameters h ¼ 6:625 Â 10 À34 Ws 2 , ¼ 193:4 THz and F ¼ 5 dB, in a relative noise power density of À157 dB=Hz, which thus makes only an insignificant contribution to the sum of the noise power densities.This calculated value was confirmed by a measurement (Table III).
In this and in the following experiment, the attenuation (Fig. 11) was adjusted so that the average power at the photodiode is P PD ¼ 0 dBm.If the input power is lowered by 20 dB to P in ¼ À17 dBm ( ) À14 dBm EDFA input power), the amplifier is no longer in saturation and an amplification of 33.7 dB sets in.After the approximation in Eq. ( 13) the noise caused by the EDFA must accordingly rise by 20 dB, which was also confirmed according to the calculation with Eq. ( 12) and the control measurement (Table III).
From this, two things follow: first, the noise contribution of the EDFA plays a negligible role in the prototype and, second, an amplifier with significantly larger maximum output power and larger amplification could be used at 6 dBm input power.In this way it is possible to use also 1 Â N splitters with N ) 16 instead of the 1 Â 16 splitter in order to extend the system.As long as the attenuation of the splitter can be compensated by the EDFA in such a way that P PD ¼ 0 dBm, the noise characteristics will not change. 7 The sum of the relative noise power densities in the transmission channel of the prototype thus amounts to AE;trans ¼ À143:9 dB Hz :

Measurement channel
In the measurement channel the noise caused by RAYLEIGH backscattering is dominant [19], page 74.To examine the noise characteristics, a semianalytical approximation was derived, based on [20,21], for the relative noise power density caused by RAYLEIGH backscattering which applies to a modulation of the optical carrier with a sinusoidal signal Ray ðfÞ % 4 FIG. 11.Noise through optical amplification.
Here, a and b are coefficients which depend on the modulation index and can be defined numerically, and Áf stands for the linewidth of the laser.R D represents the ratio between the received and transmitted optical power and can be calculated with [21], page 1795, in which R stands for the reflection factor (FBG, Fig. 6) and and L for the fiber attenuation and length.The RAYLEIGH return factor R Ray is defined as the ratio between the power P 0 fed at the entry of a fiber on the one hand and the part of the scattered power P Ray [16], page 453, which is output once again backwards at the fiber entry on the other: Here, S stands for the backscattering capture coefficient and S for the fiber attenuation caused by the RAYLEIGH backscattering.Both parameters are calculated according to [16], page 448 et seq.Assuming the following values, S ¼ 0:002 04 the spectrum of relative noise power density was calculated and shown in Fig. 12.It corresponds well with the spectrum measured in the prototype.
In Fig. 12 it can be seen that the relative noise power density is clearly dominant versus the other noise types also occurring in the measurement channel, namely receiver and shot noise as well as RIN (Table III).Since the measurement unit reduces the measurement bandwidth to B M ¼ 10 Hz, only the noise power density immediately proximate to the measurement frequency is relevant for the system It is exactly here that the correspondence of the approximation (15) with the control measurement (Fig. 12) is very good, being better than 1 dB.In the prototype a modulation index m ¼ 0:682 is used from which the coefficients a ¼ 0:935 and b ¼ 0:0646 result so that the relative noise power density is in the range of À106:9 to À118:5 dB=Hz.

B. Jitter
The noise which is added to the clock and measurement signals brings about an additive jitter.The signal-to-noise ratio (SNR) of the sinusoidal measurement signal can be calculated with and ranges from 90.6 to 102.2 dB.From this, according to [22], page 132, phase fluctuations with a standard deviation of 0.0012 maximum follow.Thus, the phase fluctuations of the measurement signal do not impair the precision of the delay measurement in Eq. (1).
System clock 1 has a rectangular shape.The standard deviation of the additive jitter t depends on the edge steepness in the zero crossing s zero and the noise power Assuming that the edge steepness is only limited by the bandwidth of the transmission system8 and this system behaves approximately like a first-order R-C9 lowpass, the step response exhibits the following temporal course At the time of the zero crossing the slope of the clock edge is This results in the time constant only depending on the bandwidth B and representing the cutoff frequency of the lowpass.We insert Eq. (25) into Eq.( 24) and obtain for the slope in the zero crossing In the prototype, with B ¼ 1 GHz, m ¼ 1, and Eq. ( 14), it amounts to 321 fs.Measurements found the slope to be greater and the noise smaller (Table III) than the calculated values.The measurement values, according to Eq. ( 21), result in a somewhat lower value.If another receiver is used having the same noise characteristics but 10 GHz bandwidth, a jitter value, according to Eq. ( 27), of 102 fs can be achieved.The jitter of the complete system is currently defined by the inherent jitter of the DDS unit and is around 7.57 ps.

C. Verification of measurement accuracy
To verify the delay measurement, the influence of the delay drifts of the receivers [6][7][8]10] had to be eliminated.For this purpose, the optical switch was located after the add/drop multiplexer of one transmission branch.At two outputs, transmission fibers (1 km and 100 m) were connected and their ends combined again by means of a coupler into one FBG (wavelength-selective reflector) to which only one receiver was connected (Fig. 13).Using a network analyzer, the transfer functions of this test setup were measured every 4 s with a 6 GHz sinusoidal signal in both switch settings.By comparing the results, the phase displacement was determined.This phase displacement was also determined by the parallel delay measurement and compared with the control measurement of the network analyzer.The result can be seen in Fig. 14; the deviation was less than AE60 fs.

D. Phase synchronization
The errors arising in phase synchronization are primarily caused by temperature-induced variations of the delays within the receivers and the DDS units.From the results of [6][7][8]10] and examinations of the DDS units, it is concluded that the average deviation in the reference time at two different reference points will be smaller than 21.2 ps.That corresponds to <0:0412 of the HF phase of the cavities, thus exceeding the required precision by more than 1 order of magnitude.
To verify this precision, the phase shift of the reference signals was measured at two different end points of the system over the period of 15 hours.The system synchronized the phases every 4 s, compensating delay fluctuations of up to 106 ps.In Fig. 15  case over one second, can be seen.The time deviation over the entire period was less than 15 ps.

VI. SUMMARY
An innovative method for distributing an item of universal time information has been presented [1].This time information consists of two reference signals which are produced phase synchronously at different spatially separate points.Reference generators derive the reference signals from two centrally generated clock signals which are sent through an optical network.This synchronization approach requires the knowledge of the clock signal delays which are regularly determined by the measurement unit.By means of an optical amplifier (EDFA) as well as the dense wavelength division multiplex (DWDM) hitherto not used in this field, it is possible to achieve very good signalto-interference ratios in transmission of both the clocks and the measurement signal.This, combined with the decoupling of the measurement system from clock transmission, make very accurate delay measurements of better than 92.6 fs possible.The prototype timing system, which has been successfully installed and tested (in preparation of a Ph.D. thesis) at GSI, is capable of providing phase-synchronous reference signals with an average accuracy of 21.2 ps, corre-sponding to <0:0412 of the rf phase of the cavities.In practical tests of the complete system, the deviation was always less than 15 ps.The short-term jitter of the reference time is 7.57 ps and is produced almost exclusively by the DDS units in the reference generators.
The described system parameters were all verified in practice.
The innovations of the presented system are the use of only one measurement unit for controlling the signal delays of all transmission lines, the use of a separate optical measurement channel, the use of DWDM for transmitting several time signals, and the compensation of delay fluctuations with DDS units.
Thanks to use of only one transmitting and one measurement unit, the system is very compact.This and the use of standard components moreover result in a very costefficient solution.

VII. PROSPECTS
The system's performance can be improved further by the following measures.
The inherent jitter of the reference generators is dependent on both the DDS unit and the frequencies of clock 1 and reference signal 1.If both frequencies are increased by a factor of 5 to f clock;1 ¼ 1 GHz and f ref;1 ¼ 250 MHz and  if another DDS unit is used, e.g.[23], jitter values of <1 ps can be achieved.Moreover, in this DDS unit the length of the phase/offset word n POW ¼ 16 bit, whereby according to Eq. ( 2) the reference time can by adjusted more precisely in increments of 61 fs.If, in addition, temperature and thus delay stabilization of the receivers is used as described in [8], the average deviation of the reference time can also be diminished significantly.By modifying the receivers and reference generators using already known technologies, it is thus possible to further significantly improve the system's performance.

FIG. 15 .
FIG.15.Time deviation of the reference signals at two different end points of the system (average interval 1 s).

FIG. 14 .
FIG.14.Time deviation between the phase displacement, determined by the delay measurement and the control measurement.

TABLE I .
Clock and reference signal frequencies.

TABLE II .
Optical carrier.
1Because of the dispersion of the fiber, the delays of the measurement signals and the clock signals are not the same.The difference between 1 and M (TableII) is about 50 ps=km.

TABLE III .
Noise in the transmission channels.
the phase shift, averaged in each