Sterile neutrinos as a possible explanation for the upward air shower events at ANITA

The ANITA experiment has observed two unusual upgoing air shower events which are consistent with the $\tau$-lepton decay origin. However, these events are in contradiction with the standard neutrino-matter interaction models as well as the $\rm EeV$ diffuse neutrino flux limits set by the IceCube and the cosmic ray facilities like AUGER. In this paper, we have reinvestigated the possibility of using sterile neutrino hypothesis to explain the ANITA anomalous events. The diffuse flux of the sterile neutrinos is less constrained by the IceCube and AUGER experiments due to the small active-sterile mixing suppression. The quantum decoherence effect should be included for describing the neutrino flux propagating in the Earth matter, because the interactions between neutrinos and the Earth matter are very strong at the EeV scale. After several experimental approximations, we show that the ANITA anomaly itself is able to be explained by the sterile neutrino origin, but we also predict that the IceCube observatory should have more events than ANITA. It makes the sterile neutrino origin very unlikely to account for both of them simultaneously. A more solid conclusion can be drawn by the dedicated ANITA signal simulations.


Introduction
The cosmic rays with energy around the Greisen-Zatsepin-Kuzmin (GZK) cutoff (∼ 50 EeV) would be strongly suppressed due to the interactions with the cosmic microwave background [1,2], i.e., p + γ CMB → p (or n) + nπ, p + γ CMB → ∆ + (1232) → p + π 0 (or n + π + ), where n is the total number of the produced π's. The ultrahigh energy (UHE) neutrinos at the scale of EeV could be copiously produced by the subsequent decays of secondary charged pions and neutrons [3]. The EeV neutrino remains to be detected, and the Antarctic Impulsive Transient Antenna (ANITA) experiment [4] is dedicated to the detection of these cosmogenic neutrinos.
In 2016, the ANITA experiment has reported one unusual steeply upward-pointing cosmic ray event 3985267 with shower energy around 0.6 EeV during the ANITA-I flight [5]. This shower event has the characteristics of the decay of a τ -lepton, which is emerging from the surface of the ice with the zenith angle around 63 • † , and the τ -lepton should be interpreted as the product of a parent ν τ by the charged-current (CC) interactions with the Earth matter. However, such a hypothesis is strongly disfavored due to that the Earth CC attenuation coefficient is 4 × 10 −6 for the neutrinos coming from such a steep angle [5]. The associated event number around E ν ∼ 1 EeV is negligible after adopting the IceCube bound on the diffuse EeV neutrinos [6], approximately E 2 ν dΦ ν /dE ν 2 × 10 −8 GeV · cm −2 s −1 sr −1 . In addition, there should be more Earth-skimming events than the steep events. The situation is worse after the ANITA detector observed the second such air shower event 15717147 with energy around 0.56 EeV at a steeper zenith angle ∼ 55 • during the ANITA-III flight [7]. Possible explanations for the anomalous events including the large transient point-source flux [7], the transition radiation of the Earth-skimming neutrinos [8], the sterile neutrino origin [9], and the decay of the quasi-stable dark matter in the Earth's core [10] have been investigated in the literature.
After the report of the first anomalous event, Ref. [9] has proposed that the sterile neutrinos could be the origin of such an event. The sterile neutrinos are well motivated by several particle physics issues and experimental anomalies. The heavy sterile neutrinos can explain the mass generation of light neutrinos through the seesaw mechanism [11][12][13][14], the sterile neutrino in the keV mass range is a good candidate of the warm dark matter [15], and the anomalies of the short baseline experiments like LSND and MiniBOONE, the Gallium source experiments as well as the reactor neutrino experiments hint at the existence of the eV-scale sterile neutrinos [16][17][18][19][20][21]. To explain the ANITA anomalous events, we need a strong sterile neutrino flux. The sources of the flux could be the superheavy dark matter decays [22][23][24][25][26][27], the topological defects [27] or some exotic interaction [28][29][30][31][32][33][34] which converts active neutrinos into sterile neutrinos during their propagation. When the sterile flux goes through the Earth, they will experience a suppressed cross section due to the small active-sterile mixing. In such a way, the neutrinos can make their way to the thin interaction region below the ANITA detector, finally producing the τ -lepton by the CC interaction with the ice, water or rock inside the interaction region. However, according to the analysis of Ref. [9], the sterile origin is in mild tension with the steep emergence angle, † The reported emergence angle of the event 3985267 is ∼ 27 • below the horizontal, the corresponding zenith angle is thus ∼ 63 • . One should be aware that the ANITA horizon is around 6 • below it's horizontal because of it's altitude. e.g. only 10% of the events are expected to emerge with the zenith angle smaller than 63 • for an active-sterile mixing angle θ = 0.1. Obviously, the second event 15717147 reported later [7] with zenith angle 55 • sharpens the tension.
In our work, the sterile neutrino origin is reexamined, and we mainly have two treatments different from Ref. [9] which are addressed as follows: • The neutrinos will lose coherence when strongly interacting with the ambient matter. After the sterile neutrino mass eigenstate flux entering the Earth, the matter will frequently measure the neutrino states such that the survived flux will collapse into the sterile state ν s .
• Because a positive detection is made only when the payload of ANITA is covered by the induced impulse cones with angle around 1 • [7], only a very small fraction of the plane flux from each direction can be detected. Thus the effective area A eff (Ω) should be much smaller than the expectation of Ref. [9].
This work is organized as follows. In section 2, we investigate the evolution of the sterile neutrinos propagating in matter with the decoherence effect included, then the angular dependence is studied. In section 3, we give our predictions of the ANITA events for different sterile neutrino parameters based on several assumptions and approximations of the experimental setup. In section 4, we make our conclusion.

Propagation of Sterile Neutrinos
Regardless of the EeV sterile neutrino sources, the sterile neutrinos will lose their coherence after a long distance of galactic travelling. They will form the mass eigenstate fluxes ν 4 and ν 1 with fractions of cos 2 θ and sin 2 θ, respectively, where θ is the active-sterile mixing angle, ν 1 harmlessly represents the three active neutrino mass eigenstates, and ν 4 can carry the mass at keV scale. When the ν 4 flux propagates into the Earth, the Earth matter will frequently interact with, or equivalently measure, the neutrino's flavor. The ν 4 -state is the superposition of the active and sterile components, i.e. ν 4 = sin θ ν a +cos θ ν s , and only the active component ν a ‡ is able to collide with the ambient matter through the CC or neutral current (NC) interactions. The matter will serve as the quantum discriminator which can resolve the mixing, making the ν 4 -state to collapse into either ν a -state or ν s -state, we refer the reader to [36][37][38] for more details. To properly take the decoherence effect into account, we adopt the following evolution equation for ν 4 (t) = c τ ν τ + c s ν s , where U is the 2 × 2 active-sterile mixing matrix with the mixing angle θ, m 1 is the averaged active neutrino mass which is negligible compared with m 4 at keV scale, 2 is due to the NC matter effect with G F being the Fermi constant, Y e ‡ Since the overlaps of ν e -ν 4 and ν µ -ν 4 are strongly constrained, the left available active component would be [35] and the references therein for recent constraints on the sterile neutrinos mixing.  the fraction of electrons and n N the nucleon number density of the matter, and L atten is the local attenuation length of the neutrino. L atten depends on the nucleon density and the neutrino energy through L atten = [σ(E ν )n N ] −1 . The number density profile of the Earth can be found in the PREM model [39]. The NC and CC cross sections are referred to [40], and we note that both the CC and NC interactions contribute to the attenuation effect. We have neglected the regeneration effects for simplicity. The τ -lepton produced by the CC can decay back to ν τ . For the NC interaction, the produced neutrino carries averagely 80% of the initial energy, but not removed from the flux. Thus our simulation will be more conservative than the realistic case. The initial conditions for the evolution read c τ (0) = sin θ, c s (0) = cos θ before the ν 4 flux entering the Earth.
Before turning to numerical demonstration of the evolution, we can first have some analytical observations. If we ignore the oscillation terms, i.e. the first two terms in the right-hand side of Eq. (1), the evolution is trivial. The active and sterile components will evolve independently such that the active component is quickly absorbed by the Earth with only the unobservable sterile component left, and there will be null signal in the detector as in the standard case. However, the sterile and active neutrinos are actually mixed, and can oscillate from one to the other if the propagation length covers the oscillation length of 2 41 ]. For the ANITA events 3985267 and 15717147 with emitting zenith angles of 63 • and 55 • , the corresponding chord lengths are 5785 km and 7309 km respectively, assuming a spherical Earth shape. Therefore, it's quite evident that ν 4 mass should be around the keV scale or even larger to convert the ν s flux into the ν τ flux when traversing the Earth. In such a way, the ν τ flux can be regenerated and survive the attenuation of the Earth. For the keV-sterile neutrinos, the Mikheyev-Smirnov-Wolfenstein (MSW) resonance condition inside the Earth can be fulfilled. However, the associated total flavor conversion effect at the resonance point as in the discussion of solar neutrino problem is not available here. The matter term in the Earth mantle for the EeV neutrino reads A NC ≈ 0.1 keV 2 . To satisfy the resonance condition A NC = ∆m 2 41 cos θ for the antineutrinos with mixing angle θ, the m 4 should be around 0.3 keV [ √ cos θ] −1 . The effective mass-squared difference when the resonance is achieved reads ∆m 2 41 = ∆m 2 41 sin 2θ ≈ 0.2 sin θ keV 2 , which corresponds to an oscillation length of [12370/ sin θ] km ∼ R/ sin θ with R = 12742 km being the diameter of the Earth. Note that the density of the Earth changes very rapidly, by 20% for the Earth mantle of 2000 km. It's very unlikely for those neutrinos to stay around the resonance while developing the phase. Our numerical calculation has included the matter effect without making any approximations.
In Fig. 1, we show the evolution of the EeV neutrino fluxes with respect to the travelling distance in the Earth. With the zenith angle of θ z = 55 • , the corresponding chord length through the Earth matter is around 7309 km. The dashed curve demonstrates the attenuation effect for the standard active neutrinos. Ref. [5] has given the Earth attenuation factor as 4 × 10 −6 for the event 3985267. Our numerical results of the attenuation factor are 1. To simplify our calculation, we assume that the initial ν 4 flux has almost monochromatic energy around 1 EeV. After these neutrinos entering the Earth, they can propagate almost freely to the other side of the Earth just as the m 4 = 2 keV case in Fig. 1. With a ν τ residue in the interaction region with depth around tens of kilometer below the ANITA balloon, the flux is eventually transformed into observable τ -lepton flux by the CC interaction. We define the efficiency of the initial ν 4 particles transformed into τ -lepton as where E 0 = 1 EeV is the initial neutrino energy, Φ ν 4 stands for the isotropic flux of neutrinos, Φ τ (E min , E max ) is the produced τ -lepton flux in the energy range of [E min , E max ] when they arrive at the Antarctic surface. We set E min = 0.2 EeV and E max = 1 EeV for our calculation, so that the two observed ANITA events are covered by [E min , E max ]. The transforming efficiency (Ω) § As has been mentioned before, Ref. [5] has only considered the CC interaction in their estimation of the attenuation length. We have included the NC interaction for the conservative purpose. Note that the actual attenuation factor should be larger after the regeneration effect included. Note that θ z 80 • is nearly above the ANITA horizon. The knee around θ z = 30 • is due to the large density jump between the Earth outer core and mantle. The energy loss of τ -leptons are simulated with the ASW model [41,42].
measures the fraction of neutrinos being converted into the observable τ -leptons during their way to ANITA, it should be noted that (Ω) is direction-dependent. In Fig. 2, we show the angular dependence of the (Ω) for the cases with m 4 = 2 keV. The dashed curve in the bottom-right corner shows the efficiency in the standard scenario. The τ -leptons induced by the standard isotropic active neutrino flux should be concentrated around the large zenith angles, therefore the Earth-skimming shower should dominate the events as has been expected. The red curve with active-sterile mixing angle of θ = 0.1 is almost uniformly distributed for the entire zenith angle range. The efficiency is around 10 −4 , which means that no matter which angle the neutrino flux comes from, there will always be one observable τ -lepton emitted after 10 4 ν 4 neutrinos entering the Earth. The Earth-skimming events don't have much advantage over the steep events in this case. However, as the active-sterile mixing angle increases, the sterile neutrino would become not so sterile due to the large mixing with the active neutrino. The efficiency tends to converge into the standard case.

ANITA Events Estimation
Due to the small Cherenkov ring angle of the EeV τ -decay shower, only a very small fraction of the τ -lepton flux obtained in the last section can be captured by the antennas of ANITA. The angle of the light cone should be around 1 • [7], so we expect the geometric area of ANITA should be about A gm ≈ 2π(D × 1 • ) 2 with D being the distance from the ANITA payload to the initial point of the shower. For event 15717147 with θ z = 55 • , we obtain the geometric area as ∼ 7.5 km 2 , slightly larger than the estimation of the ANITA group ∼ 4 km 2 [7]. The realistic geometric area estimation requires the dedicated Monte Carlo simulation, and we simply assume a constant geometric area of 4 km 2 for all emergence angles to proceed our estimation. Simulations show that the τ -lepton decay shower from larger zenith angle would have smaller impulse power [43], thus the steep shower seems more likely to be detected by ANITA than the Earth-skimming shower in the realistic case. The flux of the EeV ν 4 is bounded by the IceCube observation as dΦ ν 4 /dΩ 2 × 10 −15 [0.1/ sin θ] 2 cm −2 s −1 sr −1 , note that the flux limit is relaxed by the mixing angle suppression. The final event is obtained as where A ANITA gm (Ω) is simply fixed to 4 km 2 as mentioned before, T ANITA is the three months of exposure for ANITA, (Ω) is obtained with m 4 = 2 keV, θ = 0.1 in the last section, the flux takes the saturated value of the IceCube bound dΦ ν 4 /dΩ = 2 × 10 −15 [0.1/ sin θ] 2 cm −2 s −1 sr −1 . One can identify the effective area as A eff (Ω) = (Ω)A ANITA gm (Ω) ≈ 10 7 cm 2 , much smaller than the estimation A eff ≈ 10 11 cm 2 of [9]. Event number of 0.9 is obviously consistent with the ANITA observation. Let's check the situation for other experiments. The IceCube has a geometric area around 1 km 2 for the through-going track events, therefore we can estimate the event number of IceCube with six years observation as E IC = 6, using A IC gm (Ω) ≈ 1 km 2 , T IC ≈ 6 years. As has been pointed out in [10,44], IceCube might already have observed one such τ -track event with energy 0.1 EeV and emergence angle of 11.5 • below the horizon. The deposited energy of this track is (2.6 ± 0.3) PeV [45], implying a µ-lepton track with energy 10 PeV or a τ -lepton track with energy 0.1 EeV. No matter whether this event is an EeV τ -lepton track or not, the IceCube observation is in considerable tension with ANITA under the sterile hypothesis. The non-observation of EeV neutrino events at AUGER shouldn't be a problem with viewing angle of only a few degrees below the horizon for the Earth-skimming events [46], while the produced τ -lepton flux can be almost uniformly distributed as in Fig. 2.
In Fig. 3, we plot the relation between the ANITA's three months of event number and the active-sterile mixing angles. The mass of ν 4 is chosen to be 2 keV, and the results don't differ much for m 4 > 2 keV. The solid black curve is the total event number. The red dashed curve is the events within the zenith angle range of [0 • , 70 • ], which is larger than the blue one, i.e., the events within the zenith angle range of [70 • , 84 • ] for θ 0.1. Some comments on the figure are in order: • Note that the event number curves in Fig. 3 can be extrapolated towards very small mixing angles, but the associated sterile neutrino flux is also required to be stronger. The flux should stay around the IceCube upper limit dΦ ν 4 /dΩ ≈ 2 × 10 −15 [0.1/ sin θ] 2 cm −2 s −1 sr −1 .
• The observations of ANITA and IceCube can't be well fitted at the same time by tuning the flux and parameters of the sterile neutrino. Independent from the properties of the sterile neutrino, the τ -lepton decay events of ANITA and the through-going τ -lepton track events of IceCube always have the following relation: The unlikelihood that ANITA has two events while IceCube has one or null event can reach 3σ. Note that this estimation can also be applied to the scenario in [10], where the quasistable dark matter decay inside the Earth is the origin of the ANITA anomalous events.
A more reliable result depending on the dedicated Monte Carlo simulations of the ANITA experiment is beyond the scope of the present work.

Conclusion
In this work, we have reinvestigated the possibility of using sterile neutrino origin to explain the ANITA anomalous events. We find that the quantum decoherence effect is very important to account for the propagation behavior of EeV neutrino flux. The ν τ flux can be regenerated by the oscillation of ν s -state during their propagation inside the Earth. For the sterile neutrinos with m 4 1 keV, the Earth is almost transparent to the ν τ component in the ν 4 flux. In this way, the neutrinos can losslessly reach the interaction volume below the ANITA payload with very steep angles. We have estimated the ANITA and IceCube event number, and find that averagely the ANITA experiment is able to observe one event during the three months of exposure, while the IceCube is supposed to detect six events for it's six years of data taking. To resolve the ANITA anomaly itself, the flux of ν 4 should be around the upper IceCube limit dΦ ν 4 /dΩ ≈ 2 × 10 −15 [0.1/ sin θ] 2 cm −2 s −1 sr −1 . We have scanned the whole sterile neutrino parameter space, and find that the requirements on the sterile neutrino parameters are m 4 1 keV, θ 0.1. If the dark matter decays are the sources of the sterile neutrinos, the mixing angle shouldn't be too small, because it might be too difficult for the dark matter decays to produce so much strong neutrino flux that saturates the IceCube bound. In our framework, the predicted EeV τ -lepton track event number of IceCube is always averagely six times of the τ -lepton decay shower event number of ANITA, i.e., E IC /E ANITA ≈ 6. This result is independent of the sterile neutrino parameters as well as whether the regeneration effect is included or not. Even though there is an O(0.1 EeV) track candidate for the IceCube observation, these two experiments together are in strong tension with the sterile neutrino explanation. However, we expect the dedicated ANITA simulations to draw a more solid conclusion.