New constraints on radiative seesaw models from IceCube and other neutrino detectors

Dark matter (DM) scattering and its subsequent capture in the Sun can boost the local relic density, leading to an enhanced neutrino flux from DM annihilations that is in principle detectable at neutrino telescopes. We calculate the event rates expected for a radiative seesaw model containing both scalar triplet and singlet-doublet fermion DM candidates. In the case of scalar DM, the absence of a spin dependent scattering on nuclei results in a low capture rate in the Sun, which is reflected in an event rate of less than one per year in the current IceCube configuration with 86 strings. For singlet-doublet fermion DM, there is a spin dependent scattering process next to the spin independent one, which significantly boosts the event rate and thus makes indirect detection competitive with respect to the direct detection limits imposed by PICO-60. Due to a correlation between both scattering processes, the limits on the spin independent cross section set by XENON1T exclude also parts of the parameter space that can be probed at IceCube. Previously obtained limits by ANTARES, IceCube and Super-Kamiokande from the Sun and the Galactic Center are shown to be much weaker.


I. INTRODUCTION
Two of the biggest current challenges in theoretical particle physics concern the nature of dark matter (DM) as well as the generation of neutrino masses, both of which cannot be convincingly explained by the Standard Model (SM) [1,2]. Radiative seesaw models, as considered in this paper, connect these two open ends. In these models, neutrino masses are generated at one-loop level through interactions with a dark sector containing a DM candidate, the most famous example being the Scotogenic Model [3][4][5]. The topology of the neutrino loop as well as the particle content of the dark sector allow for many different options, which have been systematically categorized in Ref. [6].
The link between neutrinos and the dark sector allows for neutrino signals produced by DM annihilations. Such annihilations in the galactic DM halo can lead to high energy neutrinos that, if abundant enough, can be observed with IceCube and other neutrino detectors. The characteristics of the neutrino spectrum will depend on the annihilation processes involved. In particular, direct annihilation into neutrino pairs will result in a clear, distinct line at E ν m DM , whereas neutrinos produced from the decays of other SM particles created by annihilations result in a continuous spectrum. Direct annihilation into neutrinos has been studied in Refs. [7][8][9][10]. Here, we will study the general case, where both scenarios are taken into consideration.
Monochromatic neutrinos provide a clear and distinct signal compared to a continuous spectrum. Since we also take into account the latter type of signals, we consider the case where the annihilation rate is enhanced by a local boost in the relic density. We focus on annihilations due to an increased relic density in the Sun. Other astrophysical objects that have been considered in the literature are e.g. the Earth [11,12], the Galactic Center [13][14][15][16], or super massive black holes [7]. As our solar system moves through the galactic halo, DM in the form of a weakly interacting massive particle (WIMP) can scatter off nuclei in the Sun. If enough of the kinetic energy of the WIMP is transferred in these scatterings, the WIMP will be gravitationally captured by the Sun [17,18]. This will lead to an accumulation of WIMPs in the Sun's core, enhancing the local relic density and leading to a boost in annihilations. The capture rate depends chiefly on the WIMP-nucleon scattering cross section, on which direct and indirect detection experiments have put stringent constraints [1,2,[19][20][21][22][23][24][25].
For definiteness, we consider here the radiative seesaw model T1-3-B with α = 0 [26,27], following the notation in Ref. [6], that can contain either a scalar or a fermionic DM candidate. In Sec. II we give an overview of the model and in Sec. III of the WIMP-nucleon scattering processes that are present. Our method to compute the corresponding neutrino flux and event rate in IceCube is discussed in Sec. IV. In Sec. V we present the results of a numerical scan of the model parameter space. We show results for the spin independent and spin dependent WIMP-nucleon cross sections, the thermally averaged cross section in the Galactic Center and the expected event rates from annihilations in the Sun in the current IceCube configuration with 86 strings (IC86). These results are not only compared to limits from the direct detection experiments XENON1T and PICO-60, but also to previously obtained limits from neutrino observations in the Sun and the Galactic Center with ANTARES, IceCube and Super-Kamiokande. Our conclusions and an outlook are given in Sec. VI. Analytic results for mixing in the fermion sector and the resulting interaction vertices are deferred to the Appendix.

II. THE MODEL
The radiative seesaw mechanism at one loop allows for 35 models that can simultaneously generate neutrino masses and present a suitable DM candidate [6]. What these models have in common is that the new fields are odd under a Z 2 symmetry, whereas the SM fields have an even charge. As a result there must be an even number of new fields at any interaction vertex. Thus the new particles cannot decay into SM particles, which guarantees the stability of the DM candidate. Scattering and annihilation processes of new particles are allowed.
When two different dark particles are close in mass, coannihilation processes can become relevant [4,[28][29][30][31][32]. Moreover, the Z 2 symmetry does not allow for any tree-level seesaw mechanism. Instead neutrino masses are generated at one-loop level. The models in Ref. [6] are categorized based on the different topologies of the neutrino loops. Figure 1 shows the loop of T1-3 topology. In this paper we study the model T1-3-B with α = 0. The fields that are added to the SM in this case are listed in Tab. I. Componentwise these fields are given by Topology of neutrino mass generation in the T1-3 class of radiative seesaw models [6,26,27].
where the index i denotes the generations of the scalar triplet and the superscripts denote the electric charge. The scalar triplet has no hypercharge, therefore its neutral component is a real scalar. The two fermion doublets are combined to a vector-like doublet by building Dirac spinors as follows: However, throughout this work we stick to using Weyl spinors instead of these Dirac spinors.
The particle content of this model is similar to the F m 4 and S r 6 models considered in Ref. [9], except for the important difference that our fermions are mixed singlet-doublet states.
Following the notation of Refs. [26,27], the Lagrangian of the model is given by 1 The λ 6 term couples the SM leptons L i to the new fields, which allows the neutrinos to obtain their masses. Since M 2 φ > 0, the scalar triplets does not aquire a vacuum expectation value (VEV). They couple to the SM Higgs H through the λ 1 term and have self interactions through the λ 3 term. The self interactions do not influence the phenomenology of the model.
Hence λ 3 is set to zero in this work. The λ 4 and λ 5 terms are similar to Yukawa terms and link the new fermions to the SM Higgs field. After electroweak symmetry breaking (EWSB) these terms will appear in the mass matrix of the fermions and induce mixing between the fermion singlet and the vector-like doublet.
To obtain the physical states (mass eigenstates), the mass matrices must be diagonalized.
After EWSB, the mass matrix for the neutral fermions is given by and it is diagonalized by the unitary mixing matrix U χ . This results in three Majorana mass eigenstates with masses m χ 0 i , given by  Due to the interaction of the scalar triplet with the SM Higgs boson, the scalar mass matrix obtains a contribution proportional to the SM Higgs VEV v, The scalar mass matrix is diagonalized by O η , which yields the squared masses of the scalar components m 2 The mass eigenstates are defined by In this work we choose M 2 φ and λ 1 to be diagonal and thus neglect mixing between the two generations of scalar particles, as this does not affect the phenomenology. Note that the neutral and charged components have equal masses at tree level. Loop corrections induce a mass splitting between both components, making the charged component 166 MeV heavier at one loop [33].
The neutrino loop in Fig. 1 can be calculated analytically resulting in the following formula for the neutrino mass matrix: Here, n s is the number of neutral scalars and n f is the number of neutral fermions. In our model with two generations of a real scalar triplet, n s = 2. This implies that the rank of the matrix M ν ≤ 2. Thus this model only allows for two massive neutrinos. One could consider the case with three massive neutrinos, which requires a minimum of n s = 3. For our model with two massive neutrinos, Eq. (8) can be inverted leading to the Casas-Ibarra parameterization [34] λ im m ν j are the eigenvalues of the neutrino mass matrix M ν , and U PMNS the PMNS matrix [1].
R is a 3 × 2 matrix fulfilling the condition meaning that R can be parameterized as by an angle θ, which is allowed to take any value between 0 and 2π.

III. SPIN INDEPENDENT AND SPIN DEPENDENT CROSS SECTIONS
After discussing the particles in the model, as well as the generation of neutrino masses, it is now time to turn to the interactions with SM particles through scattering. In order for DM to be captured in the Sun, it needs to lose energy by scattering off nuclei inside the Sun.
The spin dependent (SD) and spin independent (SI) cross sections determine the scattering rate of WIMPs on nucleons. Experiments such as XENON1T and PICO-60 search for signals from the nuclear recoil of a DM-nucleon scattering, and they have put stringent bounds on the cross sections of these processes [1,2,[19][20][21][22]. Both the SD and SI cross sections determine the capture rate of DM in the Sun. Therefore, they have a large impact on the neutrino flux from DM annihilations. The model we consider here contains a scalar as well as a fermionic DM candidate, both of which have different scattering phenomenology.
In our case, scalar DM can only scatter via the SI process through the Higgs boson.
The SD scattering contribution with a Z 0 -boson as mediator is not present here, resulting in a lower cross section. As we will show later, this has a clear effect on the IceCube event rate. For fermionic DM there is one diagram contributing to the SI and SD cross sections each [35]. The diagram for spin (in)dependent scattering is a t-channel diagram with a (Higgs-) Z 0 -boson as mediator as illustrated in Fig. 2. Scattering processes through s-channel diagrams are not possible, since none of the new particles couples to quarks. As it turns out, the SI and SD cross sections are correlated with each other, as both are dependent on the mixing between the fermion singlet and doublet, governed by λ 4 and λ 5 . The explicit Feynman rules for the DM-mediator vertices as well as the impact of singlet-doublet mixing on the cross sections are discussed in App. A.

IV. DETECTING NEUTRINOS FROM DM IN THE SUN WITH ICECUBE
In order to determine the event rate in IceCube, we start from the differential neutrino flux at Earth, which is given by with the distance Earth-Sun d , the annihilation rate Γ χχ , the branching fractions Br ff of final annihilation particle states f , and the corresponding differential neutrino energy spectra dN f dEν . This neutrino flux is related to the differential expected signal event number by the relation where t e is the exposure time and A νµ(νµ) is the muon neutrino (muon antineutrino) effective area of the detector.
To evaluate the expected event rate numerically, we implement a new routine in mi-crOMEGAs 5.0.8 [36] that computes the expected number of signals in the IceCube detector configuration IC86. A routine based on the now obsolete configuration with 22 strings (IC22) was already implemented, so that we could use it as a basis for our new routine. We modify it by convoluting the neutrino spectra with the effective area for IC86 instead of IC22. Eight DeepCore strings are part of the IC86 configuration. Including their effective area lowers the energy threshold to 10 GeV. We extrapolate the data points linearly to fit our energy range, as shown in Fig. 3. The corresponding data points have been taken from Ref. [24]. In the region where both selections overlap, we use the IceCube effective area, as it is larger than the one of the DeepCore selection.
For deep-inelastic scattering of neutrinos with nucleons, the energy-dependent cross sections for ν µ andν µ are different. The combined effective area was split up into individual effective areas by taking these differences into account and with data from Ref. [37] and the In Tab. II we show the parameters for a typical benchmark point. This benchmark point gives an expected event rate at IceCube of 16 events per year. Fig. 4 shows the differential neutrino and antineutrino fluxes, which differ due to absorption (also oscillation  λ 6 , restricted to be small by the neutrino masses and LFV (see below) [26]. However, a continuous signal spectrum as shown in Fig. 4 is indeed expected from the Sun. For the benchmark point chosen above, the SI cross section is σ p (SI) = 1.75 · 10 −8 pb and already excluded by the XENON1T limits, but as we will see in the following, this statement does not generalize.

V. NUMERICAL RESULTS
We explore the parameter space of our model by means of a numerical scan. We use SPheno 4.0.3 [39,40] to calculate the masses of the new particles and the branching ratios (BRs) for Lepton Flavor Violation (LFV). The SPheno module is generated using SARAH 4.14.0 [41]. The relic density, spin (in)dependent cross sections and the neutrino flux from the Sun are subsequently calculated using micrOMEGAs 5.0.8 [36].
We now scan over the parameter space of the model. The mass parameters M Ψ,ψψ ,φ are varied between 100 GeV and 10 TeV. As was mentioned before, M 2 φ is chosen to be diagonal. The couplings |λ 1,4,5 | are varied between 1 · 10 −3 and 10, where λ 1 is chosen to be diagonal. The signs of these couplings are chosen randomly. λ 6 is calculated using the Casas-Ibarra parameterization Eq. (9), which requires the other model parameters as input.
The neutrino mass differences and the PMNS angles are varied in the 3σ ranges [42], where we assume Normal Ordering. The angle θ from the Casas-Ibarra parameterization is varied shown is the neutrino floor [46]. from zero to 2π. We require the relic density to be Ωh 2 = 0.12 [43], allowing it to vary by ±0.02. In addition we require the Higgs mass to be (125 ± 2.5) GeV. Lepton Flavor Violation (LFV) constrains the parameter space further. We impose the current limits on BR(µ → eγ) < 4.2 · 10 −13 [44] and BR(µ → 3e) < 1.0 · 10 −12 [45], as they impose usually the most stringent constraints. As a cross check, we have reproduced the results shown in Figs. 7 and 10 of Ref. [26].

A. Spin independent scattering
Our spin independent cross section results for both fermion (below about 1 TeV) and scalar (above about 1 TeV) DM are shown in Fig. 5. Taking into account a slightly different spread of the points due to differing scan ranges and the already imposed LFV limits, the results agree with those in Figs. 7 and 10 of Ref. [26]. The sampled points are color coded according to the dominant annihilation branching ratio. A point is marked as having no dominant channel when no single branching ratio reaches 50%.
The scalar DM candidates, all located around 2 TeV, mainly decay into a pair of Wbosons. For fermionic DM the situation is mixed. At masses below the W boson mass, the main channel is through bb production, after which the dominant channel becomes W + W − .
For masses above the top mass, there is mainly tt production, except for the points around 1 TeV, where W -boson production is the only dominant channel. This can be explained by the singlet-doublet nature of the fermionic DM, where the parameter points located around 1 TeV are those that have a large doublet contribution and therefore couple more strongly to the electroweak gauge bosons. In contrast, the points for which the DM candidate is mainly a singlet with only a small doublet admixture couple less to the gauge bosons and relatively more to the top quark via the SM Higgs boson. The charged fermions are always heavier than 102 GeV, so that the limits by the LEP experiments [47] do not restrict the parameter space. We see that the previous limits by the ANTARES [23], IceCube [24] and Super-Kamiokande [25] collaborations from DM annihilations in the Sun are several orders of magnitude weaker than the XENON1T [19] direct detection bound, with the bb limits being less stringent compared to the W + W − ones. We also show the atmospheric and diffuse supernova background (DSNB) neutrino "floor" [46], which may render DM direct detection difficult.

B. Spin dependent scattering
In Fig. 6 we show the spin dependent cross section for singlet-doublet fermion DM in our model and compare it to direct and indirect detection limits. Different annihilation channels are again color coded as before. For certain channels, ANTARES [23] and IceCube [24] impose stronger constraints than XENON1T [20], while the full data set of PICO-60 [22] has similar sensitivity as the indirect detection experiments. In a combined analysis, IceCube and PICO-60 have removed the Standard Halo Model assumption and published velocity independent limits following the suggestion in Ref. [49], which are however significantly weaker [50]. Also shown is the neutrino floor for bb final states due to high-energy neutrinos from cosmic-ray interactions with the solar atmosphere, which may render indirect detection difficult [48] and leaves little room for the bb channel at low mass beyond the PICO-60 limits.  Fig. 7 shows the thermally averaged cross section σv for the same points with fermionic and scalar DM as in Fig. 5. Here, we assume the NFW DM halo profile [52]. The expectation for a thermal relic is indicated by a dashed line [51]. For fermionic DM, it represents an upper limit, while for scalar DM it is rather a lower limit. All points are several orders of magnitude below the bounds established by IceCube [13], ANTARES [14] and their combination [15] as well as by Super-Kamiokande [16], meaning that these measurements do not constrain the model. For heavy scalar DM, the IceCube ANTARES sensitivity must be improved less than for lighter fermion DM, i.e. by only a few orders of magnitude. m DM [GeV] Super-Kamiokande [16] exclusion limits as a function of the DM mass for both fermion (below about 1 TeV) and scalar DM (above). All points are colored according to the main annihilation channel, provided there is one with a branching ratio of over 50%. Also shown is the expected cross section for a thermal relic [51].

D. Expected IceCube event rates
In Fig. 8 we show the expected event rate at IceCube for all model points of our numerical scan. For the triplet scalar DM case (green points), the spin dependent cross section is always zero. Thus in our model the accumulation of scalar DM in the Sun is only determined by the spin independent cross section, which lies below the current XENON1T bound. As can be seen, this leads to less than one event per year in the current IceCube configuration with 86 strings (IC86), whose sensitivity would therefore have to be improved by a few orders of magnitude.   neutrons, which occurs rarely in the Sun. For indirect detection, we have always used the limits for the main annihilation channel. As expected, viable models (blue) lie in particular below the rates excluded previously by IceCube (black x symbols). They can reach rates of up to ten events per year at IC86, making indirect detection competitive with respect to the direct detection limits imposed in particular by PICO-60 (orange triangles). A considerable fraction of the parameter space with high rates is excluded by the limits on the spin independent cross section set by XENON1T (red squares) due to the correlation of the spin dependent and spin independent cross sections through λ 4 and λ 5 (see App. A).
Note, however, that the correlation of spin dependent and spin independent cross sections is absent in fine-tuned scenarios where the relation between λ 4 and λ 5 is fixed as described in the Appendix.

VI. SUMMARY AND OUTLOOK
To summarize, we have studied in this paper the prospects to probe radiative seesaw models with neutrino signals from DM annihilation and detectors such as ANTARES, Super-Kamiokande and in particular IceCube, focusing on the model T1-3-B with α = 0 with either scalar triplet or singlet-doublet fermion DM. Both DM candidates can in principle directly annihilate into neutrinos. However, the relevant Yukawa couplings involved are usually strongly constrained to be small from neutrino masses and LFV processes, which are always present in these models. A sharp neutrino line at an energy corresponding to the DM mass is therefore not expected.
Continuous neutrino spectra are, however, produced from DM annihilation into decaying SM particles such as W and Z bosons, b and t quarks as well as (at least in principle) µ and τ leptons. When boosted through DM accumulation in celestial bodies such as the Earth, the Sun or the Galactic Center, the rates are observable in neutrino telescopes.
Focusing on the most promising case of the Sun, we performed a detailed analysis of the expected event rates in IceCube. In the case of scalar triplet DM, there exists no spin dependent scattering process, and the spin independent scattering cross section is too small to obtain enough accumulation inside the Sun. Because of this, the event rate of neutrino signals in IceCube would lie below one event per year, making scalar triplet DM currently undetectable with neutrino telescopes.
For singlet-doublet fermionic DM, the situation is different. In this scenario, the DM candidate can scatter via both the spin independent as well as the spin dependent process, leading to rates of up to 1000 events per year. Through our approximation of the fermion mixing matrix for small Yukawa couplings λ 4,5 , we showed then that the DM-mediator vertices in the spin (in)dependent processes, i.e. with Z 0 (Higgs) bosons, both depend on these Yukawa couplings, so that the two scattering processes become correlated. This was confirmed in a numerical scan and resulted in a considerable fraction of the parameter space with large event rates being excluded by the stringent XENON1T limits on the spin independent cross section. Previously obtained results by ANTARES, IceCube and Super-Kamiokande from the Sun and the Galactic Center were instead found to be much weaker. Constraints on the spin dependent cross section from PICO-60 and previous analyses by IceCube and ANTARES limited the viable models further to event rates of up to ten per year, leaving indirect detection with the IceCube neutrino telescope still competitive with respect to direct detection experiments.
Our results generalize to models with either real scalar triplet DM or fermion DM with singlet doublet mixing only, where scattering in the Sun is governed by similar relationships.

ACKNOWLEDGMENTS
We thank Carsten Rott for useful comments on the manuscript. This work has been supported by the DFG through the Research Training Network 2149 "Strong and weak interactions -from hadrons to dark matter" and by BMBF through Verbundforschung grant 05A20PM2.

Appendix A: Analytic results for singlet-doublet fermion mixing and vertices
The spin (in)dependent cross sections mainly depend on the three point vertex between two DM particles and the (Higgs-) Z 0 -boson. We focus on the fermion DM case since the spin dependent cross section for a scalar triplet is always zero and its coupling to the Higgs boson is given by λ 1 . The vertices for the mass eigenstates of the fermions can be calculated by SARAH [41]: Since the three mass eigenstates are generally not mass degenerate, only the elastic case where g = g , with χ g being DM, contributes to the spin (in)dependent cross section. Both vertices depend on the mixing matrix U χ between the fermion singlet and doublet. Since the singlet-doublet mixing is induced by λ 4,5 as can be seen from the fermionic mass matrix in Eq. (4), the mixing matrix U χ depends on λ 4,5 .
Diagonalizing the fermionic mass matrix proves to be difficult and gives quite unwieldy results. We can however, expand the problem for small λ 4 , and then (keepingλ = λ 5 λ 4 fixed) use perturbation theory to diagonalize this matrix approximately. At second order in λ 4 we obtain for the mixing matrix and for the diagonalized mass matrix One can see that in the case M Ψ < M ψψ (assuming λ 2 4,5 v 2 < M 2 ψψ − M 2 Ψ ) 2 , χ 1 is the lightest fermion whereas in the case M ψψ < M Ψ , χ 3 is the lightest one. Now we can use the result for U χ to calculate how the vertices depend on λ 4,5 and the mass parameters M Ψ , M ψψ . We expand the results again for small λ 4,5 , omitting all terms that contain higher orders than 2 Otherwise a more careful analysis is required. E.g. if λ 4 λ 5 > 0 or λ 4,5 < λ 5,4 MΨ M ψψ , the statement above is also true. λ 2 4 , λ 2 5 or λ 4 λ 5 . The results are: we see some deviations, which become larger for larger |λ 4 |. This is not surprising since we expanded the vertex factors for small λ 4,5 and thus our formulas are only correct for From Eqs. (A7)-(A12) it is clear that, except from special cases such as |λ 4 | = |λ 5 |, both cross sections scale with the larger of the two Yukawa couplings λ 4,5 . This yields a correlation between the spin dependent and spin independent cross section. This correlation can be made explicit using the data from our scan in Sec. V as shown in Fig. 10. This relation explains how the limits on the spin independent cross section set by XENON1T also indirectly restrict the spin dependent cross section.