Coloured Scalars Mediated Rare Charm Meson Decays to Invisible Fermions

We consider effects of coloured scalar mediators in decays $c\to u \, {\it invisibles}$. In particular, in these processes, as invisibles, we consider massive right-handed fermions. The coloured scalar $\bar S_1\equiv (\bar 3, 1, -2/3)$, due to its coupling to weak singlets up-quarks and invisible right-handed fermions ($\chi$), is particularly interesting. Then, we consider $\tilde R_2 \equiv (\bar 3, 2, 1/6)$, which as a weak doublet is a subject of severe low-energy constraints. The $\chi$ mass is considered in the range $(m_K - m_\pi)/2\leq m_\chi \leq (m_D - m_\pi)/2$. We determine branching ratios for $D\to \chi \bar \chi$, $D\to \chi \bar \chi \gamma$ and $D\to \pi \chi \chi$ for several $\chi$ masses, using most constraining bounds. For $\bar S_1$, the most constraining is $D^0 -\bar D^0$ mixing, while in the case of $\tilde R_2$ the strongest constraint comes from $B\to K {\it missing\, energy}$ . We find in decays mediated by $\bar S_1$ that branching ratios can be $\mathcal B(D\to \chi \bar \chi)<10^{-8}$ for $m_\chi=0.8$ GeV, $\mathcal B(D\to \chi \bar \chi \gamma) \sim 10^{-8}$ for $m_\chi=0.18$ GeV, while $\mathcal B(D^+ \to \pi^+ \chi \bar \chi )$ can reach $ \sim 10^{-8}$ for $m_\chi=0.18$ GeV. In the case of $\tilde R_2$ these decay rates are very suppressed. We find that future tau-charm factories and Belle II experiments offer good opportunities to search for such processes. Both $\bar S_1$ and $\tilde R_2$ might have masses within LHC reach.


I. INTRODUCTION
Low-energy constraints of physics beyond Standard Model (BSM) are well established for down-like quarks by numerous searches in processes with hadrons containing one b or/and s quark. However, in the up-quark sector, searches are performed in top decays, suitable for LHC studies, while in charm hadron processes at b-factories or/and τ -charm factories. Recently, an extensive study on c → uνν appeared in Ref. [1], pointing out that observables very small in the Standard Model (SM) offer unique (null) tests of BSM physics. Namely, for charm flavour changing neutral current (FCNC) processes, severe Glashow-Iliopoulos-Maiani (GIM) suppression occurs. The decay D 0 → νν amplitude is helicity suppressed in the SM. The authors of [2] made very detailed study of heavy meson decays to invisibles, assuming that the invisibles can be scalars or fermions with both helicites. They found out that in the SM branching ratio B(D 0 → νν) = 1.1 × 10 −31 . Then the authors of [3] found that the decay width of D 0 → invisibles in the SM is actually dominated by the contribution of D 0 → νννν. These studies' main message is that SM provides no irreducible background to analysis of invisibles in decays of charm (and beauty) mesons. They also suggested [2], that in searches for a Dark Matter candidate, it might be important to investigate process with χχγ in the final state, since a massless photon eliminates the helicity suppression. We also determine branching ratios for such decay modes. The authors of Ref. [1] computed the expected event rate for the charm hadron decays to a final hadronic state and neutrino -anti-neutrino states. They found out that in experiments like Belle II, which can reach per-mile efficiencies or better, these processes can * Electronic address:svjetlana.fajfer@ijs.si † Electronic address:anja.novosel@ijs.si be seen. In addition future FCC-ee might be capable of measuring branching ratios of O(10 −6 ) down to O(10 −8 ), in particular D 0 , D + (s) and Λ + c decay modes.
On the other hand, the Belle collaboration already reached bound of the branching ratio for B(D 0 → invisibles) = 9.4 × 10 −5 and the Belle II experiment is expected to improve it [4]. The other e + e − machines as BESSIII [5] and future FCC-ee running colliders at the Z energies [6,7] with a significant charm production with B(Z → cc) 0.22 [7] provide us with excellent tools for precision study of charm decays.
In this work we focus on the particular scenarios with coloured scalars or leptoquarks as mediators of the invisible fermions interaction with quarks. The coloured scalar might have the electric charge of 2/3 or −1/3 depending on the interactions with up or down quarks. Instead of using general assumption on the lepton flavour structure from [1] and justifying Belle bound from [8], we rely on observables coming from the D 0 −D 0 oscillations and in the case of weak doublets, we include constraints from other flavour processes.
Motivated by previous works of Refs. [1,2,[9][10][11][12], we investigate c → uχχ with χ being a massive SU (2) L singlet. Coloured scalars carry out interactions between invisible fermions and quarks. Namely, leptoquarks usually denote the boson interacting with quarks and leptons. However, the stateS 1 does not interact with the SM leptons and, therefore, it is more appropriate to call it coloured scalar. Our approach is rather minimalistic due to only two Yukawa couplings and the mass of coloured scalar. The effective Lagrangian and coloured scalar mediators are introduced in Sec. II. In Sec. III we describe effects ofS 1 mediator in rare charm decays, while in Sec. IV we give details ofR 2 mediation in the same processes. Sec. V contains conclusions and outlook. In experimental searches, the transition c → u invisibles might be approached in processes c → u / E with / E being missing energy. Therefore, invisibles can be either SM neutrinos or new right-handed neutral fermions (having quantum numbers of right-handed neutrinos), or scalars/vectors as suggested in Ref. [2]. The authors of Refs. [1,9] considered in detail general framework of New Physics (NP) in c → u invisibles, relying on SU (2) L invariance and data on charged lepton processes [9]. They found that these assumptions allow upper limits as large as few 10 −5 , while in the limit of lepton universality branching ratios can be as large as 10 −6 . To consider invisible fermions, having quantum numbers of right-handed neutrinos, and being massive, we extend the effective Lagrangian by additional operators as described in Refs. [3,13] In Ref. [1] right-handed massless neutrinos are considered. Also, in Ref. [12] authors considered charm meson decays to invisible fermions, which have negligible masses. In the following, we consider massive righthanded fermions and use further the notation ν R ≡ χ R . Following [13], we write in Table I interactions of the coloured scalarS 1 andR 2 with the up quarks andR 2 and S 1 with down quarks.
Cloured Scalar Invisible fermion S1 = (3, 1, Table I. The coloured scalarsS1, S1 andR2 interactions with invisible fermions and quarks. Here we use only right-handed couplings of S1. Indices i, j refer to quark generations.
We concentrate only on coloured scalar and scalar leptoquark due to difficulties with vector leptoquarks. Namely, the simplest way to consider vector leptoquarks in an ultra-violet complete theory is when they play the role of gauge bosons. For example, U 1 is one of the gauge bosons in some of Pati-Salam unification schemes [14,15]. However, other particles with masses close to U 1 with many new parameters in such theories, making it rather difficult to use without additional assumptions.
Coloured scalars contributing to transition c → uχχ have following Lagrangians, as already anticipated in in [13] L(S 1 ) ⊃ȳ RR Here, we give only terms containing interactions of quarks with right-handed χ. The S 1 scalar leptoquark, in principle, might mediate c → uχχ on the loop level, with one W boson changing down-like quarks to u and c. Obviously, such a loop process is also suppressed by loop factor 1/(16π 2 ) and G F making it negligible. Also, due to the right-handed nature of χ, one can immediately see that in the case ofS 1 , the effective Lagrangian has only one contribution with In the case ofR 2 with For the mass of χ, kinematically allowed, in the c → uχχ decay, one can relate this amplitude to b → sχχ or in s → dχχ. However, it was found [16] that the experimental rates for K → πνν are very close to the SM rate [17], leaving very little room for NP contributions. Therefore, we avoid this kinematic region and consider mass of χ to be m χ ≥ (m K − m π )/2, while the charm decays allow m χ ≤ (m D − m π )/2. For our further study it is very important that χ is a weak singlet and therefore LHC searches of high-p T lepton tails [18,19] are not applicable for the constraints of interactions in the cases we consider. However, further study of final states containing mono-jets and missing at LHC and future High luminosity colliders will shed more light on these processes.
III.S1 IN c → uχχ Due to its quantum numbers, the coloured scalarS 1 and χ can interact only with up-like quarks. Most generally, the number of χ's can be three and the matrix y RR can have 9 × 2 parameters. Here, we consider one χ, that can couple to both u and c quarks. These two couplings might enter in amplitudes for processes with down-like quarks at loop-level, as discussed in [20]. Obviously, due to the right-handed nature of χ, one can immediately see that in the case ofS 1 , the effective Lagrangian has only the contribution First, we discuss constraints from D 0 −D 0 mixing and then consider exclusive decays D 0 → χχ, D 0 →χχγ, and D → πχχ. The authors of Ref. [12] considered scalar leptoquarks allowing each up-quark can couple to different flavour of lepton or right-handed neutrino. In such a way, they avoid constraints from the D 0 −D 0 mixing.
The strongest constraints on χ interactions with u and c comes from the D 0 −D 0 oscillations. The interactions in Eqs. (2) and (3) can generate transition D 0 −D 0 . Coloured scalarS 1 contributes to the operator entering the effective Lagrangian [13,21] L Dmix with the Wilson coefficient given by The standard way to write the hadronic matrix ele- (27)(4) calculated in the MS scheme, which was computed by the lattice QCD [22] and the D meson decay constant defined as 0|ūγ µ γ 5 c)|D(p) = if D p µ , with f D = 0.2042 GeV [23]. Due to large nonperturbative contributions, the SM contribution is not well known. Therefore, in the absence of CP violation, the robust bound on the product of the couplings can be obtained by requiring that the mixing frequency should be smaller than the world average x = 2|M 12 |/Γ = (0.43 +0.10 −0.11 )% by HFLAV [24]. The bound on this Wilson coefficient can be derived following [20,25] with a renormalisation factor r = 0.76 due to running of C 6 from scale MS 1 1.5 TeV down to 3 GeV. One can derive The amplitude for this process can be written as giving the branching ratio (15) Using Belle bound B(D 0 → χχ) < 9.4 × 10 −5 [8], one can find easily the bound on Wilson coefficient c RR Belle < 0.046. This value is derived for the mass m χ = 0.8 GeV. We analyse the dependence on the mass ofS 1 , allowing the mass of χ to be (m K − m π )/2 < m χ < (m D − m π )/2, and assume the branching ratio for B(D 0 → χχ) < 10 −10 , 10 −9 and 10 −8 , with ȳ RR 1 cχȳ RR * 1 uχ = 1. We present our result in Fig. 2 and find that mass ofS 1 , using these reasonable assumptions, can be within LHC reach.

D 0 → χχγ
The authors of Ref. [2] suggested, that the helicity suppression, present in the D 0 → χχ amplitude for m χ = 0, is lifted by an additional photon in the final state and therefore D 0 → χχγ might bring additional information on detection of invisibles in the final state. They found that the branching decay is In the above equations x χ = m χ /m D , F DQ = 2/3(−1/(m D − m c ) + 1/m c ), f D = 0.2042 GeV [23] and Y (x χ ) is given in Appendix. Coefficient c RR is constrained by Eq. (13). Comparing these results with

D → πχχ
The rare charm decays due to GIM-mechanism cancellation are usually dominated by long distance contributions. Long distance contributions to exclusive decay channel D → πνν were considered in Ref. [26]. For example, the branching ratio BR(D + → π + ρ 0 → π + νν) < 5 × 10 −16 . The authors of [26] discussed another possibility D + → τ + ν → π +ν ν and found that the branching ratio should be smaller than 1.8 × 10 −16 . An interesting study of these effects was done in Ref. [27], implying that in order to avoid these effects one should make cuts in the invariant χχ mass square, The amplitude for D → πχχ can be written as with the standard form-factors definition with q = p − k. We follow the update of the form-factors in Ref. [28]. This enables us to write the amplitudes in the form given in Ref. [21] M(D(p) → π(k)χ(p 1 )χ(p 2 )) = with the following definitions (20) We can the differential decay rate distribution as with notation λ ≡ λ(m 2 D , m 2 π , q 2 ), (λ(x, y, z) = (x + y + z) 2 − 4(xy + yz + zx)), β = 1 − 4m 2 χ /q 2 and N = Note that in case of charged charm meson there is a multiplication by 2 in the differential decay rate compared to neutral D. The integration bounds should be 4m 2 χ ≤ q 2 ≤ (m D − m π ) 2 in the case of m χ = 0.5, 0.8, while instead of m χ = 0.18 GeV, q 2 cut is used from Ref. [27], giving the lowest mass of the invisibles should be searched in the region m χ ≥ q 2 cut /4 0.29 GeV. This enables us to avoid the region in which the effects of the long distance dynamics dominates. One can use Table IV. Branching ratios for B(D → πχχ). In the second and the third columns the constraint from the D 0 −D 0 mixing is used, assuming the mass of MS 1 = 1000 GeV. In the case mχ = 0.18, the cut in integration variable is done by taking q 2 cut , as described in the text.

IV.R2 IN c → uχχ
TheR 2 leptoquark is a weak doublet and it interacts with quark doublets (3). Therefore, the appropriate couplings,ỹ LR 2 sχỹ LR * 2 bχ can be constrained from the b → sχχ and s → dχχ decays, as well as from observables coming from the B s −B s , B d −B d , K 0 −K 0 oscillations as in [20]. We consider the most constraining bounds coming from decays B → K / E and from the oscillations of B s −B s , relevant for the χ mass region (m K − m π )/2 < m χ < (m D − m π )/2. The decay B → K / E was recently studied by the authors of Ref. [29]. They pointed out that current bound on the rate B → K / E when the SM branching ratio for B → Kνν is subtracted from the experimental bound on B(B + → K + / E) is the most constraining. They derived B(B → K / E) < 9.7 × 10 −6 as the strongest bound among B → H s / E (H s is a hadron containing the s quark).

Constraints from B → K / E and Bs −Bs oscillations
The amplitude for B → Kχχ can be written as In the case of Wilson coefficient c LR B it is easy to find [13] c LR The integration over the phase space depends on the mass of m χ we chose. Here we can choose a mass χ, which we used in D decays (m K − m π )/2 < m χ < (m D − m π )/2. The bounds on the Wilson coefficient in Eq. (23) are following |c LR B | < 3.3 × 10 −4 , < 4.9 × 10 −4 and < 9.1 × 10 −4 for m χ = 0.18, 0.50, 0.80 GeV.
There are two box diagrams with χ within the box contributing to the B s −B s oscillations. The contribution ofR 2 box diagrams to the effective Lagrangian for the B s −B s oscillation is We can understand this result in terms of the recent study of new physics in the B s −B s oscillation in [30]. The authors of [30] introduced the following notation of the New Physics (NP) contribution containing the righthanded operators as (25) Following their notation, one can write the modification of the SM contribution by the NP as in Ref. [ (26) They found that R SM loop = (1.31 ± 0.010) × 10 −3 and η = α s (µ N P )/α s (µ b ). Relying on the Lattice QCD results of the two collaborations FNAL/MILC [31], HPQCD [32], the FLAG averaging group [33] published following results, which we use in our calculations From these results, one can easily determine bound The same couplingsỹ LR 2 sχỹ LR * 2 bχ enter in the D 0 −D 0 mixing (9) and condition (12), and one can derive The bound on coefficients in (28) lead to the one order of magnitude stronger constraint then one in (29), y LR 2 sχỹ LR * 2 bχ < 1.58 × 10 −6 MR 2 /GeV. In our numerical calculations we use this bound and do not specify the mass ofR 2 . However, one can combine these constraints and determined theR 2 mass, which can satisfy both conditions. In Fig. (4) we present dependence of the couplingsỹ LR  that the largest mass ofR 2 , which satisfies both conditions is MR 2 4400, 7100, 10800 GeV for the masses m χ = 0.8, 0.5, 0.18 GeV respectively. AllR 2 masses below these limiting values are allowed and interestingly, they are within LHC reach. Using the same expressions as in the previous section, we calculate branching ratios for D 0 → χχ, D 0 → χχγ and present them in Table V Table V. Branching ratios for B(D 0 → χχ) and B(D 0 → χχγ). The bounds on the Wilson coefficient c LR D derived from the B(B → K / E) < 9.7 × 10 −6 for selected masses of χ from the range (mK − mπ)/2 < mχ < (mD − mπ)/2. mχ (GeV) B(D 0 → π 0 χχ) B(D + → π + χχ) 0.18 < 8.7 × 10 −13 < 4.5 × 10 −12 0.50 < 1.1 × 10 −12 < 5.4 × 10 −12 0.80 < 1.7 × 10 −13 < 8.7 × 10 −13 Table VI. Branching ratios for B(D 0 → π 0 χχ) and B(D + → π + χχ). The bounds on the Wilson coefficient c LR D derived from B(B → K / E) < 9.7 × 10 −6 . In the case mχ = 0.18, the cut in the integration variable is done by taking q 2 cut , as described in the text.
they are c LR D = |(V us V * cb +V cs V * ub )c LR B | = 4.4×10 −6 , 6.6× 10 −6 , 1.2 × 10 −5 . Compared with the coloured scalarS 1 mediation, the branching ratios for all three decay modes are suppressed for several orders of magnitude, indicating the important role of constraints from B mesons. Such suppressed branching ratios of the all rare charm decays mediated byR 2 is almost impossible to observe. On the other hand, decays of hadrons containing b quarks , mediated byR 2 a much more suitable for searches of invisible fermions.

V. SUMMARY AND OUTLOOK
We have presented a study on rare charm decays with invisible massive fermions χ in the final state. The mass of χ is taken to be in the range (m K − m π )/2 < m χ < (m D − m π )/2, since the current experimental results on B(K → πνν) are very close to the SM result, almost excluding the presence of New Physics. We considered two cases with coloured scalar mediators of the up-quarks interaction with χ. The simplest model is one with S 1 = (3, 1, −2/3), which couples only to weak up-quark singlets, and the second mediator isR 2 = (3, 2, 1/6) which couples to weak quark doublets.
In the case ofS 1 , the relevant constraint comes from the D 0 −D 0 oscillations. We have calculated branching ratios for D 0 → χχ, D 0 → χχγ and D → πχχ. The charm meson mixing severely constrain the branching ratio D 0 → χχ in comparison with the experimental result for the branching ratio of D 0 → / E. For our choice of m χ the branching ratio for D 0 → χχγ can be calculated using experimental bound on the rate for D 0 → / E. In this case, there is an enhancement factor up to three orders of magnitude smaller, depending on the mass of χ in comparison with the constraints from the D 0 −D 0 os-cillations. The branching ratios for D → πχχ, based on charm mixing constraint, are of the order 10 −9 − 10 −7 , suitable for searches at future tau-charm factories, BE-SIII and Belle II experiments.
In the case ofR 2 , for the mass range of χ relevant for charm meson rare decays, we rely on constraints coming from B(B → K / E) and from the B 0 s −B 0 s mixing. We find that the all three decay modes D 0 → χχ, D 0 → χχγ and D + → π + χχ are now having branching ratios for a factor 3 − 4 orders of magnitude smaller then in the case of coloured scalarS 1 mediation, making them very difficult for the observation.
Interestingly, the mass of both mediatorsS 1 andR 2 are in the range of LHC reach, and hopefully, searches for mono-jets and missing energy might put constraints on their masses.

VI. ACKNOWLEDGMENT
The work of SF was in part financially supported by the Slovenian Research Agency (research core funding No. P1-0035). The work of AN was partially supported by the Advanced Grant of European Research Council (ERC) 884719 -FAIME.