Constraining a general U ( 1 ) ′ inverse seesaw model from vacuum stability , dark matter and collider

Arindam Das, ∗ Srubabati Goswami, † Vishnudath K. N., 3, ‡ and Takaaki Nomura § Department of Physics, Osaka University, Toyonaka, Osaka 560-0043, Japan Theoretical Physics Division, Physical Research Laboratory, Ahmedabad 380009, India Discipline of Physics, Indian Institute of Technology, Gandhinagar 382355, India School of Physics, KIAS, Seoul 02455, Korea Abstract We consider a class of gauged U(1) extensions of the Standard Model (SM), where the light neutrino masses are generated by an inverse seesaw mechanism. In addition to the three right handed neutrinos, we add three singlet fermions and demand an extra Z2 symmetry under which, the third generations of both of the neutral fermions are odd, which in turn gives us a stable dark matter candidate. We express the U(1) charges of all the fermions in terms of the U(1) charges of the standard model Higgs and the new complex scalar. We study the bounds on the parameters of the model from vacuum stability, perturbative unitarity, dark matter relic density and direct detection constraints. We also obtain the collider constraints on the Z ′ mass and the U(1)′ gauge coupling. Finally we compare all the bounds on the Z ′ mass versus the U(1)′ gauge coupling plane.


I. INTRODUCTION
The discovery of the Higgs boson with a mass of 125 GeV at the Large Hadron Collider (LHC) [1,2] has placed the SM on a firm footing. However, the SM still does not have answers to some of the very fundamental questions like the origin of the neutrino masses and the existence of dark matter (DM). A straight forward way to include the generation of the sub-eV scale neutrino masses and the presence of the DM into the SM is by adding extra particles, which may or may not involve the extension of the SM gauge group.
Among the various beyond standard model (BSM) scenarios that have been proposed in the literature, the models in which the SM is extended by a U (1) gauge group has received some attention. The models with an extra U (1) gauge group naturally contain three right handed neutrinos as a result of the conditions for the gauge anomaly cancellation. Thus, the active light neutrino masses can be generated via the canonical type-I seesaw mechanism [3][4][5][6]. However, in canonical type-I seesaw model, which is considered in most of the U (1) extended models, one either has to go for extremely large Majorana masses (∼ 10 14 GeV) or very small Yukawa couplings (∼ 10 −6 ), making it difficult to probe the heavy neutrinos at the colliders. Motivated by testability in colliders, various TeV scale extensions of the type-I seesaw model have been considered in the literature (for recent reviews, see [7][8][9][10]). One of the most popular TeV scale seesaw models is the inverse seesaw model [11] where the smallness of the neutrino mass can then be attributed to a small lepton number violating term. A tiny value of this lepton number violating term is deemed natural, since when this parameter is zero, the global U(1) lepton number symmetry is reinstated and neutrinos are massless. Especially, an inverse seesaw mechanism in the context of a U (1) B−L extension of the SM has been studied in reference [12]. In these models, the presence of extra singlet fermions (in addition to the right handed neutrinos) helps us to bring down the seesaw scale (which is the U (1) breaking scale) to ∼ O(TeV), simultaneously allowing for large Yukawa couplings, Y ν ∼ O(0.1).
An important aspect of the U (1) extended models which has been scrutinized recently is the implications for the stability of the electroweak (EW) vacuum [13][14][15][16][17][18][19][20][21][22]. The measured values of the SM parameters, especially the top mass M t and strong coupling constant α s implies that there exists an extra deeper minima near the Planck scale (M P lanck ), which threatens the stability of the present EW vacuum [23,24], since this may tunnel into that true vacuum. The calculation of the decay probability suggests that the present EW vacuum is metastable at 3σ which means that the decay time is greater than the age of the universe. It is well known that the scalar couplings pull the vacuum towards stability whereas the Yukawa couplings push it towards instability. The EW vacuum stability in the context of a class of minimal U (1) extensions containing extra scalars and fermions have been studied by the authors of [16][17][18]20] and they have shown that the behaviour of the EW vacuum depends also on the U (1) quantum numbers chosen, since the renormalization group equations (RGEs) depend on these quantum numbers. The conformal symmetric versions of such models have been considered in references [21,22].
As already mentioned, the existence of the DM is another major motivation for going beyond the standard model. Measurements by Planck and WMAP demonstrate that nearly 85 percent of the Universe's matter density is dark [25]. Hence, it is very important to study models that can simultaneously explain neutrino mass as well as DM and their theoretical as well as phenomenological implications. The models with an extra U (1) gauge group can accommodate a DM candidate even in the minimal version (with type-I seesaw), by adding an additional Z 2 symmetry [26,27], where the third generation of the right handed neutrinos act as the DM candidate. Other versions of the U (1) B−L extension with scalar DM have been studied in [28][29][30].Also, there are various realizations of the grand unified theories (GUTs) that predict the existence of extra Z boson [31,32].
The presence of the extra Z boson that couples to the quarks and the leptons also gives rise to a rich collider phenomenology in the U (1) models [20,22,33,34]. Searches for such Z boson through it's decay dileptons have been conducted by the ATLAS and the CMS collaborations and lower limits on the Z mass has been obtained [35][36][37].
In this paper, we consider a class of gauged U (1) extensions of the SM, where active light neutrino masses are generated by an inverse seesaw mechanism. In addition to the three right handed neutrinos, we add three singlet fermions and demand an extra Z 2 symmetry under which, the third generations of both the neutral fermions are odd, which in turn gives us a stable DM candidate. This allows us to consider large neutrino Yukawa couplings and at the same time, keeping the U (1) symmetry breaking scale to be of the order of ∼ O(1) TeV. The main difference of this inverse seesaw model from that considered in [12] is that the extra neutral fermions that we are adding are singlets under the gauge group and hence we do not have to worry about anomaly cancellation. Also, instead of considering one particular model, we express the U (1) charges of all the fermions in terms of the U(1) charges of the SM Higgs and the new complex scalar. We perform a comprehensive study of the bounds on the model parameters from low energy neutrino data, vacuum stability, perturbative unitarity and DM as well as collider constraints. The rest of the paper is organized as follows. In sections II and III, we introduce the class of the U (1) models under consideration and discuss the fermionic and the scalar sectors. We discuss the fitting of the neutral fermion mass matrix in section IV, by taking all the experimental constraints into account. In section V, we discuss the RGE evolution of the couplings and present the parameter space allowed by vacuum stability and perturbative unitarity in various planes. This is followed by a discussion on the DM scenario in these models, where we present the parameter space giving the correct relic density and satisfying the direct detection bounds at the same time. In section VII, we discuss the combined bounds from vacuum stability, unitarity, DM relic density and the collider constraints and finally, we summarize in section VIII.

II. MODEL AND NEUTRINO MASS AT THE TREE LEVEL
The model considered is based on the gauge group SU (3) c ×SU (2) L ×U (1) Y ×U (1) . In addition to the SM particles, we have three right handed neutrinos ν Ri , a complex scalar Φ required to break the U (1) symmetry and three gauge singlet Majorana fermions S i . An extra Z 2 symmetry is imposed to have a stable fermionic dark matter. The matter and Higgs sector field content along with their transformation properties under SU (3) c × SU (2) L × U (1) Y × U (1) are given below.
S ∼ (1, 1, 0, 0). (2.4) Note that the generation indices have been suppressed here. Under Z 2 , the third generation of ν R and S, i.e., ν R3 and S 3 are odd whereas all the other particles are even and we assume that this Z 2 is not broken.
The U (1) charges of the fermions are defined to satisfy the gauge and gravitational anomaly-free conditions: The most general Yukawa Lagrangian (along with the Majorana mass for S) invariant under that could be written using the fields given above is, The invariance of this Yukawa Lagrangian under the U (1) symmetry gives us the following conditions : Using these conditions and the anomaly-free conditions, the U (1) charges of all the fermions could be determined in terms of x H and x Φ as, Note that the choice x Φ = 1 and x H = 0 correspond to the well known U (1) B−L model. From Eq.(2.6), after symmetry breaking, the terms relevant for neutrino mass are, where, M D = Y ν H and M R = y N S Φ . The neutral fermion mass matrix M ν can be defined as, The mass scales of the three sub-matrices of M ν may naturally have a hierarchy M R >> M D >> M µ . Then, the effective light neutrino mass matrix in the seesaw approximation is given by, Because of the extra Z 2 symmetry, the Yukawa coupling matrices Y ν and y N S and hence the mass matrices M D and M R will have the following textures, In addition, we will choose M µ to be diagonal without loss of generality. Since ν R3 and S 3 do not mix with other neutral fermions, they will not contribute to the seesaw mechanism and we will have a minimal inverse seesaw mechanism (3 ν L + 2 ν R + 2 S case) in which the lightest active neutrino will be massless. The two fermions ν R3 and S 3 mix among themselves and the lightest mass eigenstate could be a stable DM candidate. In the heavy sector, we will have two pairs of

III. SCALAR POTENTIAL OF THE MODEL AND SYMMETRY BREAKING
The scalar potential of the model is given by, The trivial conditions that give a stable potential are, λ 1 > 0 ; λ 2 > 0 and λ 3 > 0, (3.2) and if λ 3 < 0, the stability of the potential can still be achieved by satisfying the following conditions : The above conditions are obtained by demanding the Hessian matrix corresponding to the potential to be positive definite at large field values [16,38].
The two scalar fields acquire vacuum expectation values(vevs) given by, The values of v and u are determined by the minimization conditions and are given by, After symmetry breaking, the mixing between the fields h and φ could be rotated away by an orthogonal transformation to get the physical mass eigenstates as, The values of v and u are determined by the minimization conditions and are given by, After symmetry breaking, the mixing between the fields h and φ could be rotated away by an orthogonal transformation to get the physical mass eigenstates as, The values of v and u are determined by the minimization conditions and are given by, After symmetry breaking, the mixing between the fields h and φ could be rotated away by an orthogonal transformation to get the physical mass eigenstates as, The masses of the scalar eigenstates are, From these, one can get the relations, We use these equations to set the initial conditions on the scalar couplings λ 1 , λ 2 and λ 3 while running the renormalization group equations. Also, from the above equations, one can get, (3.11)

A. Perturbative Unitarity
In addition to the vacuum stability conditions, the constraints from the perturbative unitarity conditions also put bounds on the model parameters. By considering the hh → hh and φφ → φφ processes, one can derive combined constraints on the three couplings appearing in the scalar potential [39,40] : Demanding the other running couplings to remain in the perturbative regime gives us, where g i stands for SM gauge couplings. For the U (1) gauge coupling g , we require, (x q,d,u,l,e,ν,Φ )g , (x H /2)g < √ 4π. (3.14)

IV. NUMERICAL ANALYSIS AND PARAMETER SCANNING IN THE NEUTRINO SECTOR
To study the parameter space allowed by vacuum stability as well as perturbativity bounds up to M P lanck using the RGEs, we have to first fix the initial values for all the couplings. While setting the initial values for the neutrino Yukawa couplings Y ν and y N S , we have to make sure that they reproduce the correct oscillation parameters and satisfy all the experimental constraints. To do this, we find sample benchmark points for Y ν , y N S and M µ and the vev of the extra scalar Φ(u) by fitting them with all the constraints using the downhill simplex method [41]. Note that here,  • Cosmological constraint on the sum of light neutrino masses as given by the Planck 2018 results [43]. This puts an upper limit on the sum of active light neutrino masses to be, Note that in our case, the lightest active neutrino is massless and also we are restricting our analysis only to the normal hierarchy (NH) of the active neutrino masses since the vacuum stability, dark matter and collider analyses are independent of the hierarchy of the light neutrino masses. In addition, it has been found that the best fit of the data is for the NH and IH is disfavoured with a ∆χ 2 = 4.7(9.3) without (with) Super-Kamiokande atmospheric neutrino data [44]. Thus we have, • The constraints on the oscillation parameters in their 3σ range, given by the global analysis [42,45] of neutrino oscillation data with three light active neutrinos following NH are given in Table I. We use the standard parametrization of the PMNS matrix in which, where c ij = cosθ ij , s ij = sinθ ij and the phase matrix P = diag (1, e iα 2 , e i(α 3 +δ) ) contains the Majorana phases.
• The constraints on the non-unitarity of U P M N S = U L as given by the analysis of electroweak precision observables along with various other low energy precision observables [46]. At 90% confidence level, we have, This also takes care of the constraints coming from various charged lepton flavor violoating decays like l i → l j γ, among which µ → e γ is the one that gives the most severe bound [47], In table (II), we give two benchmark points consistent with all the experimental data discussed above. As a consistency check, we also give the value of Br(µ → e γ) obtained at the two benchmark points.
Parameter   is given as a check.

V. RGE EVOLUTION
The couplings in any quantum field theory get corrections from higher-order loop diagrams and as a result, the couplings run with the renormalization scale. We have the renormalization group equation (RGE) for a coupling C as, where i stands for the i th loop and β C is the corresponding β function.
We have evaluated the SM coupling constants at the the top quark mass scale and then run them using the RGEs from M t to M P lanck . For this, we have taken into account the various threshold corrections at M t [48][49][50]. Then the SM RGEs are used to run all the couplings up   Table II and we have fixed g = 0.1 and y 33 N S = 0.5.
whereas the one in the right side is for m h 2 = 20 TeV and θ = 0.003. For the neutrino Yukawa couplings, we have used BM-I from the Table II and     values of M Z as low as 1 TeV are allowed and correspondingly, x H is allowed from −8 to 8.

VI. DARK MATTER SCENARIO
In this section we discuss dark matter physics in our model with respect to the constraints from relic density and direct detection experiments. As mentioned earlier, the third generations of N R and S L (N 3 R , S 3 L ) are odd under the Z 2 parity in the general U (1) inverse seesaw model that we consider. This ensures the stability of N 3 R and S 3 L which is required for these to be potential DM candidates. As a result the relevant interactions in the Lagrangian can be written as Note that N 3 R can not couple to the SM Higgs and lepton doublets due to the Z 2 symmetry. After the symmetry breaking we have Φ = u √ 2 and the mass matrix can be written as, . Now rotating the basis we can write the physical eigenstates as where tan 2θ = where we take m ψ 1 < m ψ 2 . Thus ψ 1 is the lightest Z 2 odd particle and our DM candidate. Putting ψ 1 and ψ 2 back into Eq. 6.1 along with the physical mass eigenstates of h and φ we write the interaction among Z 2 odd fermion and scalars as, Then the DM candidate can annihilate through the scalar portal (Fig. 6a), where interactions between h 2 and SM particles are induced by scalar mixing (See Eq.3.8) and these couplings are equal to the SM Higgs couplings times sin θ. In addition, the DM can annihilate to the SM particles via Z exchange (Fig. 6c) where the gauge interactions are given by,

A. Relic density
Here we analyze the relic density of our DM candidate. The DM candidate ψ 1 annihilate into the SM particles via processes induced by Z and scalar boson interactions as shown in imply much lower values of M Z where the Z exchange is not a dominant process. We also find that the Z mediated process cannot provide sufficient annihilation cross section to explain the observed relic density if DM is heavier than ∼ 3 TeV, complying with the requirement that the gauge coupling satisfy (x q,d,u,l,e,ν,Φ )g , (x H /2)g < √ 4π for perturbativity. This tendency comes from the fact that the annihilation cross section is P-wave suppressed since our DM is Majorana fermion.
We will now focus on the contribution of h 2 exchange process to the relic density of DM. We fixed Z mass and g for simplicity. Note that we chose m h 2 ∼ 2M DM since we can obtain the observed relic density in this region via h 2 exchange process as discussed above. In Fig. 9, we show the allowed parameter space in M DM − y 33 N S and m h 2 − sin θ planes that give the correct relic density of DM, 0.11 < Ωh 2 < 0.13, adopting the approximate range around the best fit value [43]. From the left panel of Fig. 9, we can see that in general, for larger values of M DM , the allowed values of y 33 N S are large. But, a few points with smaller values of y 33 N S are also obtained for M DM > M Z since ψ 1 ψ 1 → h 2 → Z Z process is kinematically allowed there. In the right panel of Fig. 9, we have shown the allowed parameter space in the m h 2 − sin θ plane. From this plot, we can see that sin θ can be small for M DM > M Z (m h 2 ∼ 2M DM ) since h 2 Z Z coupling is not suppressed by sin θ as we can see from Eq. (6.7). However, we have some lower limit of sin θ for M DM < m Z since here, ψ 1 ψ 1 → h 2 → Z Z process is kinematically disallowed and the coupling of h 2 to the SM particles is suppressed by sin θ.
where the effective couplings are, Hence the effective Lagrangian can be written as, where m h 1 and m h 2 are the SM and BSM Higgs masses. The corresponding cross section of Fig. 6b in the non-relativistic limit can be calculated as, where we apply f N = 0.287 for neutron [55] 1 and v = 246 GeV. We then estimate the cross sections applying allowed parameter sets obtained in previous subsection and the results are shown in Fig. 10. The black dotted and dashed lines show the current upper bounds from PANDAX-II [53] and XENON-1t [54] respectively. We find that our parameter region is allowed by the direct detection constraints since the cross section is suppressed by small sin θ which is also preferred by the constraints from vacuum stability. The cross section will be further explored by the future direct detection experiments like XENON 1t, PandaX, etc. In this section, we consider the production of Z from the proton proton collision at the LHC and its decay into different types of leptons. We first calculate the Z production cross section at the LHC from protons followed by the decay into lepton, pp → Z → + − with = e, µ.
In our analysis we calculate the cross section combining the electron and muon final states. We compare our cross section with the latest ATLAS search [37] for the heavy Z resonance. Since we are considering U (1) models with extra Z , the ATLAS results can be compared directly with our results. Atlas analysis has considered different models like SSM and Z ψ [56] where the Z decays into e and µ. Conservatively considering these limits for our case we first produce the Z (300 GeV ≤ M Z ≤ 6 TeV) at the 13 TeV LHC followed by the decay into the dilepton mode and finally compare with the cross sections in our model. To calculate the bounds on the g , we calculate the model cross section, σ Model , for the process pp → Z → 2e, 2µ, with a U (1) coupling constant g Model at the LHC at the 13TeV center of mass energy. Then we compare this with the observed ATLAS bound (σ Observed ATLAS ) for Γ m = 3% which has been studied for the SSM. The corresponding cross sections are plotted in Fig. 11 for different choices of x H and x Φ . Thus, the value of g corresponding to a given M Z is given as,  Here also, we have fixed θ = 0.01. The green dots in Figs. 12 and 13 correspond to the values that give the correct DM relic density. The constraints coming from this is seen to be less stringent than the combined constraints from vacuum stability, perturbativity and ATLAS analysis.

VIII. CONCLUDING REMARKS
In this paper we have studied the inverse seesaw model in a class of general U (1) extensions of the SM. We have studied the parameter spaces in various planes that are allowed by vacuum stability and perturbativity as well as consistent with the low energy neutrino data. In addition, this model has a prospective DM candidate resulting from the stabilization of the third generations of the SU (2) L singlet neutral fermions using the odd parity under the discrete Z 2 symmetry. Comparing the Z production and its decay into the dilepton mode at the LHC with the current ATLAS results, we find the bounds on the U (1) coupling constant with respect to the Z mass. Finally, combining all the constraints, we obtain the resultant allowed parameter space which can be probed in the future experiments.

ACKNOWLEDGMENTS
This work of A. D. is supported by the Japan Society for the Promotion of Science (JSPS) Postdoctoral Fellowship for Research in Japan.