Consistency of a gauged two-Higgs-doublet model: Scalar sector

We study the theoretical and phenomenological constraints imposed on the scalar sector of the gauged two Higgs doublet model proposed recently as a variant of the popular inert Higgs doublet model of dark matter. The requirements of tree-level vacuum stability and perturbative unitarity in the scalar sector are analyzed in detail. Furthermore, taking into account the constraints from the 125 GeV Higgs boson measurements at the Large Hadron Collider, we map out the allowed ranges for the fundamental parameters of the scalar potential in the model.


I. INTRODUCTION
Despite the plentiful and cogent evidences of dark matter (DM) from astrophysical and cosmological observations over several decades, the identity of DM is still unknown. Whether the solution should reside on the particle physics side or belong to the pure gravity doctrine is not settled. Perhaps this dilemma will not be resolved anytime soon and will continue baffling us for a while. On the other hand, after the monumental discovery of the 125 GeV Higgs boson at the Large Hadron Collider (LHC), more data and statistics are still needed to determine whether the observed Higgs boson is the one predicted in the standard model (SM) or it is part of an extended scalar sector proposed in many models beyond the SM (BSM). Among these BSM models with additional scalars, the general two Higgs doublet model (2HDM) is possibly the most popular and well-studied class of models [1]. One of the intriguing 2HDM variants is the inert Higgs doublet model (IHDM) [2][3][4][5], where the neutral component of the second Higgs doublet, in light of an imposed discrete Z 2 symmetry on the scalar potential, is stable and hence can be a DM candidate. Over the years, many thorough phenomenological analysis of the IHDM had been carried out in the literature [6][7][8][9], while the origin of multiple inert Higgs doublets in the context of grand unification was explored in [10].
The discrete Z 2 symmetry, however, is introduced by hand without proper justification. Besides, it is commonly believed that global symmetry (regardless of being discrete or continuous) is deemed to be strongly violated by gravitational effects [11,12]. To remedy these unappealing features, we elevate the IHDM into a more theoretically elegant setting [13], in which the two Higgs doublets H 1 and H 2 comprise a doublet H ¼ ðH 1 ; H 2 Þ T of a new non-Abelian SUð2Þ H gauge group. For various phenomenological reasons, H is also charged under an additional Abelian gauge group Uð1Þ X . Gauge invariance ensures that only hH 1 i can develop a nonzero vacuum expectation value (VEV) while hH 2 i remains zero. Thus H 1 and H 2 play the role of the SM and inert Higgs doublets respectively. There is no need to impose the discrete Z 2 symmetry in the scalar potential as it can emerge as a low-energy symmetry after spontaneously symmetry breaking. We dubbed this model as gauged two-Higgs-doublet model (G2HDM) in [13]. In IHDM, either the neutral scalar or pseudoscalar component of H 2 can be a DM candidate, while in G2HDM the whole neutral component of H 2 does the job.
Phenomenology of G2HDM at the LHC have been explored previously in [13,14] for Higgs physics and in [15] for the new gauge bosons. In this work, we will focus on some theoretical issues that were not addressed before. We will analyze the scalar sector in more detail. Our main goal is to identify the allowed regions for the fundamental parameters/couplings in the scalar potential by imposing the tree-level theoretical constraints from vacuum stability and perturbative unitarity, as well as phenomenological constraints from the 125 GeV Higgs boson measurements at the LHC. Similar studies have been presented for the general 2HDM [16][17][18] and for the IHDM [6,19].
Vacuum stability comes from the requirement that the potential has to be bounded from below, while perturbative unitarity demands the amplitudes of all scattering processes in the scalar sector to lie within the unitarity circle.
In G2HDM, the VEVs of various scalars in the extended Higgs sector will induce complicated mixings among the scalars in the flavor basis. In particular, the discovered 125 GeV boson at the LHC is in general a linear combination of the three neutral components of H 1 , Φ H and Δ H , where Φ H and Δ H are the SUð2Þ H doublet and triplet respectively. These extra scalars are necessarily introduced in G2HDM for various phenomenological reasons, as explained in detail in [13]. Only in the absence of mixings, would the neutral component of H 1 be identified as the observed 125 GeV Higgs boson. As a result, we have to include these mixing effects when imposing the constraints from the measurements of the Higgs boson mass and the diphoton signal strength. The contributions to the diphoton final state from the charged Higgs bosons and new heavy fermions in G2HDM will be carefully scrutinized. There are also mixings between H 0 2 , G m H and Δ m , the lower components of H 2 , Φ H and Δ H respectively, rendering the analysis of DM physics more involved. We will discuss the extended scalar sector of G2HDM in more detail in the next section. However, the thorough study of DM phenomenology in G2HDM will be carried out in a separate work.
The content of the paper is laid out as follows. In Sec. II, we review some crucial features of the G2HDM, with a focus on the particle content and the scalar potential of the model, followed by discussions on the minimization conditions of the scalar potential and the scalar mass spectra. We will take this opportunity to investigate the impacts of two new quartic terms, namely −H † 1 H 1 H † 2 H 2 þ H † 1 H 2 H † 2 H 1 and H † Φ H Φ † H H, in the scalar potential which were missing in our previous works. In Sec. III, we study the theoretical constraints on the scalar potential parameters from tree-level vacuum stability and perturbative unitarity of the scalar sector. In Sec. IV, we further impose the LHC constraints associated with the 125 GeV Higgs boson and its diphoton signal strength on the parameter space. Finally, we summarize our findings in Sec. V.

II. G2HDM SET UP
In this section, we start with a brief review on the setup of G2HDM [13] by specifying its matter content and writing down the scalar potential including the two new terms There are different ways of introducing new heavy fermions in the model but we choose a simplest realization: the heavy fermions together with the SM right-handed fermions comprise SUð2Þ H doublets, while the SM lefthanded doublets are singlets under SUð2Þ H . We note that heavy right-handed neutrinos paired up with a mirror charged leptons forming SUð2Þ L doublets was suggested before in [20]. To render the model anomaly-free, four additional chiral (left-handed) fermions for each generation, all singlets under both SUð2Þ L and SUð2Þ H , are included. For the Yukawa interactions that couple the fermions to scalars in G2HDM, we refer our readers to [13] for more details.
To avoid some unwanted pieces in the scalar potential and construct gauge invariant Yukawa couplings, we require the matter fields to carry extra Uð1Þ X charges. Thus the complete gauge groups in G2HDM consist of SUð3Þ C ×SUð2Þ L ×Uð1Þ Y ×SUð2Þ H ×Uð1Þ X . Apart from the matter content of G2HDM summarized in Table I, 1 there also exist the gauge bosons corresponding to the SM and the extra gauge groups.
The salient features of G2HDM are: (i) it is free of gauge and gravitational anomalies; (ii) renormalizable; (iii) without resorting to the previous ad-hoc Z 2 symmetry, an inert Higgs doublet H 2 can be naturally realized, providing a DM candidate; (iv) due to the gauge symmetries, 2 dangerous flavor-changing neutral currents vanish at tree level for the SM sector; (v) the VEV of the triplet can trigger SUð2Þ L symmetry breaking while that of Φ H provides a

Matter Fields
SUð3Þ C SUð2Þ L SUð2Þ H Uð1Þ Y Uð1Þ X We note that u H L , d H L , ν H L , e H L were denoted as χ u , χ d , χ ν , χ e respectively in [13]. 2 For using Abelian Uð1Þ symmetry to avoid flavor-changing neutral currents in 2HDM at tree level, see e.g., [21,22]. mass to the new fermions through SUð2Þ H -invariant Yukawa couplings; etc.

B. Higgs potential
The Higgs potential invariant under both SUð2Þ L × Uð1Þ Y and SUð2Þ H × Uð1Þ X can be decomposed into four different terms as 3 (i) Uð1Þ X is introduced to simplify the Higgs potential V T in Eq. (1). For example, a term Φ T H Δ H Φ H obeying the SUð2Þ H symmetry would be allowed in the absence of Uð1Þ X . Note that as far as the scalar potential is concerned, treating Uð1Þ X as a global symmetry is sufficient to kill this and other unwanted terms. However having a global symmetry lying around in the model defeats our original purpose of gauging the discrete global Z 2 symmetry. Therefore we prefer to treat Uð1Þ X as a local symmetry.
(ii) Note that the μ 2 H and μ 2 Φ terms in VðHÞ and VðΦ H Þ have the "right" signs while the μ 2 Δ in VðΔ H Þ has the "wrong" sign. (iii) The quadratic terms for H 1 and H 2 have the following coefficients respectively. Since the four parameters M HΔ , λ HΔ , λ HΦ and λ 0 HΦ can take on either positive or negative values, even with a positive μ 2 H , Eqs. (8) and (9) can be negative and positive respectively so that one can achieve hH 1 i ≠ 0 and hH 2 i ¼ 0 to break SUð2Þ L . Because the doublet H 2 does not obtain a VEV, its neutral component, if lighter than the charged Higgs, can potentially be a DM candidate whose stability is protected by the SUð2Þ H gauge symmetry. (iv) Similarly, the quadratic terms for two fields Φ 1 and Φ 2 have the coefficients respectively. As in the above cases of H 1 and H 2 , even with a positive μ 2 Φ , one can achieve hΦ 1 i ¼ 0 and hΦ 2 i ≠ 0 by judicious choices of the parameters.
(v) In (4), if −μ 2 Δ < 0, SUð2Þ H is spontaneously broken by the VEV hΔ 3 i ¼ −v Δ ≠ 0 with hΔ p;m i ¼ 0 by applying an SUð2Þ H rotation. In fact, this also triggers the symmetry breaking of the other gauge symmetries as discussed in the next subsection. (vi) All terms in VðHÞ, VðΦ H Þ, VðΔ H Þ and V mix ðH; Δ H ; Φ H Þ are Hermitian, implying all the coefficients are necessarily real. Thus the scalar potential in G2HDM is CP-conserving.
(vii) Classification of all the symmetry breaking patterns of G2HDM like the analysis made in [2] for 2HDM would be very interesting but nevertheless beyond the scope of this work. We would like to address this issue in the future.

C. Spontaneous symmetry breaking
To facilitate spontaneous symmetry breaking, let us shift the fields as follows Here v, v Φ and v Δ are VEVs to be determined by minimization of the potential. The set Ψ G ≡ fG þ ; G 0 ; G p H ; G 0 H g are Goldstone bosons, to be absorbed by the longitudinal components of W þ , W 3 , W p , W 03 respectively.
Substituting the VEVs in the potential V T in Eq. (1) leads to Note that the two new couplings λ 0 H and λ 0 HΦ do not appear in Eq. (13). Thus minimization of the potential in Eq. (13) yield the same set of VEV equations as in [13]. For convenience, we here list them again One can solve for nontrivial v 2 and v 2 Φ in terms of v Δ and other parameters using Eqs. (14) and (15). Substituting these solutions of v 2 and v 2 Φ into Eq. (16) gives rise to a cubic equation of v Δ which can be solved either analytically or numerically. Once v Δ is known, one can substitute its value back to Eqs. (14) and (15) to determine v and v Φ respectively. In this way, one can explicitly see the effects of the triplet's VEV v Δ on the breaking of the SM SUð2Þ L × Uð1Þ Y and the Uð1Þ X , after it first triggers SUð2Þ H symmetry breaking.
In Sec. IV, we will study numerically if this trigger mechanism can indeed happen for μ 2 Δ > 0 and μ 2 H > 0. As pointed out before, the scalar potential of G2HDM is CP-conserving, since all the coefficients in the scalar potential are real. Since the triplet VEV v Δ satisfies a cubic equation, it may have one real and two complex solutions. The complex solutions are nevertheless unphysical and we will discard such solutions in our numerical scans described in later sections. In the cases where there are three real solutions, we will pick the one that has the lowest energy as long as it is consistent with the SM Higgs VEV v ¼ 246 GeV.

D. Scalar mass spectrum
The scalar boson mass spectrum can be obtained from taking the second derivatives of the potential with respect to the various fields and evaluating the results at the minimum of the potential. The mass matrix thus obtained contains three diagonal blocks. The first block is 3 × 3. In the basis of S ¼ fh; ϕ 2 ; δ 3 g it is given by This matrix can be diagonalized by a similarity transformation with an orthogonal matrix O, defined as jfi i ≡ O ij jmi j with jfi i and jmi j referring to the flavor and mass eigenstates respectively, where the three eigenvalues are in ascending order: pairs will all be decreased by an overall factor of O 11 , implying a reduction by jO 11 j 2 on the h 1 decay branching ratios into these channels. On the other hand, as we shall see later, h 1 → γγ (and also h 1 → Zγ) involves extra contributions from the δ 3 and ϕ 2 components, which could lead to either enhancement or suppression with respect to the SM prediction.
The second block is also 3 × 3. In the basis of G ¼ fG p H ; H 0Ã 2 ; Δ p g, it is given by It is straightforward to show that Eq. (19) has a massless eigenstate corresponding to the physical Goldstone bosonG p , which is a mixture of G p H , H 0Ã 2 and Δ p . 6 The other two eigenvalues are the masses of two physical fields D andΔ: The field D can be a DM candidate in G2HDM. Note that in the parameter space where the quantity inside the square root of Eq. (20) is very small,Δ would be nearly degenerate with D. Under this circumstance, we need to include coannihilation processes for relic density calculation. We note that, besides the scalar D, it is possible in G2HDM to have fermionic ν R , ν H L , ν H R or vector W 0ðp;mÞ to be a DM candidate as well, depending on which one is 6 The two physical Goldstone fields the lightest. In this work, we will assume D is the DM candidate. In our numerical scan, detailed in later sections, we will check to make sure D must be lighter than W 0ðp;mÞ , H AE and all heavy fermions.
The final block is 4 × 4 and diagonal with for the physical charged Higgs H AE , and for the four Goldstone boson fields G AE , G 0 and G 0 H . Note that we have used the minimization conditions Eqs. (14)- (16) to simplify various matrix elements of the above mass matrices. If the charged Higgs mass is close to the DM mass, we should include the corresponding coannihilation contributions as well.
The six Goldstone particles G AE , G 0 , G 0 H andG p;m will be absorbed by the longitudinal components of the massive gauge bosons W AE , Z, Z 0 and W 0ðp;mÞ . It implies that there are two unbroken generators and thus two massless gauge particles left over after spontaneous symmetry breaking. One is naturally identified as the photon while the other one could be interpreted as a dark photon γ D or another neutral gauge boson Z 00 . To give a mass to the γ D or Z 00 , one can either use the Stueckelberg mechanism [23][24][25][26] or introduce yet another Higgs field Φ X solely charged under Uð1Þ X to break one of the remaining two unbroken generators. Depending on the magnitude of the Stueckelberg mass M X or the VEV hΦ X i, one can identify the extra neutral gauge boson as either γ D or Z 00 . Only one unbroken generator for the massless photon should remain at the end of this game. The physical neutral gauge bosons γ, Z, Z 0 and γ D =Z 00 are in principle mixtures of the gauge field components W 3 , B, W 03 and X [13].
After symmetry breaking, by scrutinizing the whole Lagrangian, one can discover that an effective Z 2 parity can be assigned consistently to the physical particle spectrum of the model: All the SM particles (with h 1 identified as the 125 GeV Higgs observed at the LHC), Z 0 , γ D =Z 00 , h 2 and h 3 are even, while D,Δ, H AE , W 0ðp;mÞ as well as all heavy fermions f H are odd under this accidental discrete symmetry. As mentioned above, we will assume D is the lightest odd particle and can serve as a DM candidate in this work.

III. THEORETICAL CONSTRAINTS
In this section, we will discuss the theoretical constraints arising from tree-level vacuum stability (VS) and perturbative unitarity (PU) on the scalar sector in G2HDM.

A. Vacuum stability
For the stability of the vacuum we have to examine the scalar potential at large-field values and make sure it is bounded from below. Therefore it is sufficient to consider all the quartic terms in the scalar potential which are Following the methods in [27,28], we introduce the following basis (x, y, z) and two ratios ξ and η, defined as One can show that the ratios satisfy 0 ≤ ξ ≤ 1 and −1=4 ≤ η ≤ 0. While η is always above −1=4, the actual lower bound and its effects will be discussed in more detail below. To deduce the conditions for the potential to be bounded from below, we rewrite the quartic terms V 4 in terms of x, y and z with the ratio parameters ξ and η. That is andλ H ðηÞ ≡ λ H þ ηλ 0 H ,λ HΦ ðξÞ ≡ λ HΦ þ ξλ 0 HΦ . According to [29,30], using the Sylvester's criterion or requiring semipositive definite eigenvalues of the quadratic form Qðξ; ηÞ, albeit mathematically rigorous, overly constrain the parameter space in this case. Instead, the notion of copositivity (conditionally positive) criteria was proposed for vacuum stability conditions. This is because the Sylvester's criterion or positive semidefinite requirement applies to the case that the basis (x, y, z) can be either positive or negative, whereas copositivity is applicable to the situation of positive (or non-negative) (x, y, z) only. As our x, y and z are the square of the scalar fields and thus nonnegative, we will use the copositivity criteria accordingly. 7 The copositivity criteria for the 3 × 3 symmetric matrix Qðξ; ηÞ are [29,30] with 0 ≤ ξ ≤ 1 and −1=4 ≤ η ≤ 0 as stated before. As shown in [29], these copositivity criteria are necessary and sufficient conditions for the quartic potential to be bounded from below. It is easy to see that for a given set of λ parameters, the conditions above are monotonic functions of the ratios ξ and η. If this last two ratios were perfectly independent, it would be enough to check Eqs. (32)-(34) only at the boundaries of said ratios. However, in our case there is a correlation between the lower bound of η and the value of ξ given by η min ¼ ξðξ − 1Þ. Since this constraint limits the value of η to be well inside the rectangle defined by the original boundaries mentioned above, checking the corners of the rectangle would actually yield stronger constraints on the scalar potential parameters. Instead, we should check the boundary defined by η min ¼ ξðξ − 1Þ for Λ HΦ ðξ; η min Þ and Λ HΦΔ ðξ; η min Þ.
We apply the method outlined in [32] to Λ HΦ ðξ; η min Þ and Λ HΦΔ ðξ; η min Þ to find the actual conditions to apply. In particular, we need to check the minimum values of Λ HΦ ðξ; η min Þ and Λ HΦΔ ðξ; η min Þ, when necessary, in the boundary mentioned above. Consider the first derivative of Λ HΦ ðξ; η min Þ with respect to ξ at the boundary defined by By solving dΛ HΦ ðξ; η min Þ=dξ ¼ 0, we find 8 the minimum ξ 0 For this ξ 0 to be inside the (0,1) range, the absolute value of the second term must be smaller than 1=2. This results in the condition ðλ 0 If ξ 0 is not in the (0,1) range we can skip the following checks.
If ξ 0 is inside the (0,1) range we can check that it is actually a minimum by finding the sign of the second derivative, given by By substituting ξ ¼ ξ 0 we find that the sign of the second derivative is given by the sign of λ 0 H . If λ 0 H is negative, then Λ HΦ ðξ; ξ 2 − ξÞ is concave and it is enough to check the boundary points ðξ; η min Þ ¼ ð0; 0Þ and (1,0). But if λ 0 H is positive then we have a minimum and we have to check if Λ HΦ ðξ 0 ; ξ 2 0 − ξ 0 Þ ≥ 0. In summary, to make sure that Λ HΦ ðξ; ηÞ ≥ 0 we have to check: 7 For other rigorous method to deduce the necessary and sufficient conditions for various scalar potentials in BSM to be bounded from below, see e.g., [31]. 8 There are actually two solutions, differing only in the sign in the second term. However, the solution with positive sign is a solution only when Λ HΦ ðξ; η min Þ is concave and ξ 0 is actually a maximum. and only in the cases that both conditions λ 0 H > 0 and ðλ 0 HΦ Þ 2 λ H < ðλ 0 H Þ 2 λ Φ apply we also need to check: This same procedure can be applied to Λ HΦΔ ðξ; η min Þ defined by Eq. (34) but the resulting expressions are far more complicated and add little to nothing to the discussion. However we follow the same procedure and, in additional to the boundary points at ðξ; ηÞ ¼ ð0; 0Þ and (1,0), we also check the corresponding minimum when needed.
For the rest of the conditions, if such a minimum exists, it corresponds to the point ðξ; ηÞ ¼ ð1=2; −1=4Þ, which we also check together with the (0,0) and (1,0) boundary points.

B. Perturbative unitarity
To infer the tree-level perturbative unitarity constraints on the scalar potential parameters, similar to the discussion of vacuum stability above it is sufficient to focus on the quartic couplings. The reason is that at high energies the 2 → 2 scatterings induced by either the scalar cubic couplings or gauge interactions are suppressed by the propagator (which is inversely proportional to momentum transfer squared) compared to those induced by the quartic couplings which do not have such a momentum dependence. We will follow the same procedures adopted in [17][18][19]33] that deduce the perturbative unitarity constraints for the SM, 2HDM and IHDM.
In terms of the physical and Goldstone bosons defined in Eq. (12), the quartic terms V 4 can be written as We now analyze all the 2 → 2 scalar scatterings induced by the above quartic potential.
(I) The scattering amplitudes with the initial and final states containing one of the following states hh ffiffi ffi can be straightforwardly computed based on Eq. (40). 9 The corresponding amplitude matrix M 1 in the basis of Eq. (41) is 9 We have included in the amplitude a symmetry factor of 1= ffiffi ffi 2 p for identical particles in the initial and final states.
where the ði; jÞ element corresponds to the amplitude between the states of the ith and jth element of the basis; for instance, the (1,1) element represents the amplitude of the process hh → hh.
HΦ . There are 10 eigenvalues in total and seven of them are given by: whereλ þ H ≡ λ H þ λ 0 H =2. The rest of the eigenvalues are the three roots of the equation which can be solved either numerically or analytically. Since at very high center-of-mass energies all masses can be ignored, one can take ffiffi ffi s p → ∞. The 2 ↔ 2 scatterings among the scalars are then governed by the quartic couplings. The amplitudes are all constants without energy dependence and we can focus on the s-wave [17]. Unitarity constraints then require all the 10 real eigenvalues λ i of the Hermitian M 1 to satisfy [17][18][19]33] jλ i ðM 1 Þj ≤ 8π; ∀ i ¼ 1; …; 10: There also exist processes involving different particles in the initial and final states that can be divided into the following groups: (II) For the basis fhH 0Ã 2 ; G 0 H 0Ã 2 ; G þ H − g as initial and final states, we have the following scattering matrix with eigenvalues 2λ H and 2λ H AE λ 0 H . (III) For the basis fhG þ ; H 0Ã 2 H þ ; G 0 G þ g, we find the scattering matrix is the same as that in Eq. (45). (VI) (VIII) (IX) The scattering amplitudes of the following processes and are all equal to λ HΦ . (X) The scattering amplitudes of the following processes and are all equal toλ HΦ . (XI) (XII) The scattering amplitudes of the following processes are all equal to λ HΔ .
(XIII) The scattering amplitudes of the following processes are all equal to λ ΦΔ . To summarize: For the above 13 groups of scattering processes, perturbative unitarity requires the following constraints ðXIIÞ; ðXIIIÞ ⇒ jλ HΔ j ≤ 8π; jλ ΦΔ j ≤ 8π:

C. Numerical results from vacuum stability and perturbative unitarity
In this section, we will present numerical results from the constraints of VS and PU. The VS constraints correspond to Eqs. (32)- (34) in Sec. III A, i.e., the constraints on the copositivity of the matrix Qðξ; ηÞ. On the other hand, the PU constraints can be found in Sec. III B and are summarized in Eq. (60). For the two ratios ξ and η defined in Eqs. (28) and (29), we will use the endpoint values ξ ¼ 0, 1 and η ¼ −1, 0 in our analysis. In the following, λ H;Φ;Δ and λ 0 H are referred to as the diagonal couplings, and λ HΦ;HΔ;ΦΔ and λ 0 HΦ as the off-diagonal couplings. In the upper-and lower-left panels of Fig. 1, the allowed regions for diagonal couplings λ Φ and λ Δ versus offdiagonal couplings λ HΦ and λ HΔ respectively are presented.
In fact, the allowed regions projected on to the λ H -λ HΦ (λ H -λ HΔ ) plane behave quite similarly to those of λ Φ -λ HΦ (λ Δ -λ HΔ ). The green and blue regions are the allowed regions by VS and PU respectively, while only the red regions are allowed when both VS and PU are considered. Clearly, the constraints from PU [Eq. (60)] alone will impose upper limits on all diagonal couplings λ i , where i ¼ ðH; Φ; ΔÞ, as shown by the blue regions for the cases of λ Φ and λ Δ . On the other hand, the green regions from the constraints of VS Eqs. (32)-(34) restrict the off-diagonal λ HΦ and λ HΔ to be greater than −2 for the diagonal λ i < 0.1. In contrast, for larger λ i , the windows for negative values of λ HΦ and λ HΔ are widened.
The small wedgelike regions bounded by the green, blue and red contours with negative λ HΦ;HΔ and 0.1 ≲ λ H ; λ Φ ≲ 4 are allowed by either VS or PU alone. However as both the VS and PU constraints are imposed, this region is excluded as the two constraints are in tension. We found that this tension is mainly caused by a nontrivial combination of the copositive conditions in Eq. (33) and the unitarity constraints in Eq. (60). Loosely speaking, if the off-diagonal λ ij is negative, then larger λ i and λ j are needed as indicated in Eq. (33). However, the PU conditions set limits on the size of the diagonal couplings jλ H;Δ;Φ j which in turn implies lower limits on the off-diagonal λ ij via the VS constraint in Eq. (33). The last VS constraint of Eq. (34) actually does not yield much more information since the only couplings therein that can be negative areλ HΦ ðξÞ, λ HΔ and λ ΦΔ , whose lower limits are already constrained by Eq. (33).
In the upper-and lower-right panels of Fig. 1 (32). Similar to the two left panels, the small wedgelike regions surrounded by blue and red contours are excluded when the both VS and PU constraints are taken into account.
In Fig. 2, we present the allowed regions for the offdiagonal terms λ ij s. Obviously, smaller λ ij s are always allowed by all the theoretical constraints. The limits of λ ij s obtained from the PU conditions are jλ HΦ j ≲ 18, jλ 0 HΦ j ≲ 18, jλ ΦΔ j ≲ 15, and jλ HΔ j ≲ 11. Interestingly, λ HΔ is slightly more stringent than others because it has relatively large coefficients inside the scattering matrix M 1 in Eq. (42). On the other hand, effects of the VS condition are hardly visible in the parameter space except for the λ HΦ − λ 0 HΦ plane. The constraints originate from the parameterλ HΦ ðξÞ. Recall thatλ HΦ ðξÞ ≡ λ HΦ þ λ 0 HΦ ξ with 0 ≤ ξ ≤ 1. From the first inequality of Eq. (33)λ HΦ ðξÞ ≥ −2 , it implies λ HΦ and λ 0 HΦ cannot be too negative at the same time, leading to the blank bottom-left corner in the bottomleft panel of Fig. 2. As already explained in Fig. 1, when we combine both constraints from VS and PU, the allowed region is further shrunk due to the tension between the copositive conditions Eq. (33) and the unitarity limits Eq. (60).

IV. HIGGS PHENOMENOLOGY
In this section, we will consider the phenomenological constraints from the Higgs physics at the LHC.

A. Higgs diphoton decay
In G2HDM, as explained above the 125 GeV Higgs boson h 1 is a linear combination of h, ϕ 2 and δ 3 : where O ij are the elements of the orthogonal matrix O that diagonalizes the mass matrix M 2 0 displayed in Eq. (18). The mixing has an impact on both the ggH production and Higgs decay branching ratio into two photons. Due to the narrow Higgs decay width, the Higgs production will be dominated by the resonance region and thus the cross section σðpp → h 1 → γγÞ can be well approximated by [34] σðgg → h 1 → γγÞ ¼ with the center of mass energy ffiffi ffi s p ¼ 13 TeV and the integral of the parton (gluon in this case) distribution function product evaluated at the scale of μ ¼ m h 1 ¼ 125 GeV. The corresponding signal strength for ggH production is with N C being the number of color and . The symbol f refers to the SM fermions while F denotes the heavy fermions. The form factors for spins 0, 1 2 and 1 particles are given by with the function fðτÞ defined as fðτÞ ¼

<
: The parameters τ i ¼ m 2 h 1 =4m 2 i with i ¼ H AE , f, F, W AE are related to the corresponding masses of the particles in the loops. Other symbols not defined in Eq. (65) are selfexplanatory. As is well known in the SM, even if the W gauge boson or top quark becomes infinitely heavy (τ i → 0), they do not decouple in the triangle loop and have still finite contributions due to A 1=2 ð0Þ ¼ 4=3 and A 1 ð0Þ ¼ −7. Moreover, h SM → γγ is dominated by the W AE loop contribution, while the top quark contribution is subdominant and has destructive interference with that of W AE . In G2HDM, the extra contributions from charged Higgs and new heavy charged fermions to the diphoton channel can be either constructive or destructive interferences with the SM ones, depending on the signs of C h and O 21 =O 11 respectively. However, due to an extra factor of 1=m 2 H AE in front of A 0 ðτ H AE Þ and the small ratio v=v Φ in front of A 1=2 ðτ F Þ in Eq. (65), both the contributions from the very heavy charged Higgs and extra fermions are not significant in the diphoton channel. They are effectively decoupled in the large mass limit in G2HDM.
On the other hand, the partial decay width of h 1 into two gluons mediated by the SM quarks and the new colored fermions is [35][36][37] Depending on the sign of O 21 =O 11 , the contributions from the new heavy quarks in G2HDM can increase or decrease the branching fraction. Note also the suppression factor of the small ratio v=v Φ for the contributions from new heavy quarks. Finally, the SM prediction for the branching ratios of h → gg and h → γγ can be easily obtained from the previous formulas by setting O 11 ¼ 1 and O 21 ¼ O 31 ¼ 0 and neglecting the contributions from the new heavy fermions and H AE .

B. Numerical results from Higgs physics
In this subsection, we will discuss constraints on the scalar quartic couplings λs from Higgs physics only (marked in orange) and from Higgs physics (HP) plus the aforementioned theoretical conditions (marked in magenta). We also colored in red the theoretical constraints alone as before. Here, we include the HP experimental constraints in the 2σ range from the ATLAS experiment [39]: (ii) The signal strength of the Higgs boson produced through gluon-gluon fusion decaying to two photons, μ γγ ggH ¼ 0.81 þ0.19 −0.18 . In addition, other theoretical criteria are also implemented: (i) the charged Higgs H AE and the extra gauge boson W 0ðp;mÞ must be heavier than the DM particle D, (ii) any point with negative scalar mass squared is discarded, and (iii) the vacuum must be a global minimum. Since λ 0 H contributes to the charged Higgs mass and the h 1 H þ H − coupling through mixing effects but not to the neutral Higgs mass spectra, its constraint from the current LHC Higgs data is coming from the charged Higgs running inside the triangle diagram of diphoton channel, which is quite loose, in particular when the charged Higgs mass is getting very heavy.
To deduce the constraints from HP, we will adopt the allowed ranges for all the λ-parameters from the theoretical VS and PU constraints in Eqs. (32)-(34) and (60). Additional parameters needed to be considered in this subsection are M HΔ , M ΦΔ and the VEVs (v, v Φ , v Δ ). We will fix v ¼ 246 GeV as the SM Higgs VEV but v Φ will be set to 10 TeV in order to make the new gauge bosons heavy and thus satisfy the bounds from LHC high-mass resonances searches [40,41]. The third VEV v Δ will take values in the range f0.5; 20g TeV and the two parameters M HΔ and M ΦΔ will be varied in the range f−1; 1g TeV. Once these parameters have been chosen, the minimization conditions Eqs. (14)-(16) will determine the numerical value of the μ 2 i parameters. As mentioned in Sec. II B, we require μ 2 Δ > 0 to break SUð2Þ H which induces SUð2Þ L breaking. To verify this, we examine the sign of μ 2 H given the previous parameter choices. It turns out that for those points which satisfy all the constraints, 40% of them have positive μ 2 H . It implies that SUð2Þ H breaking in some cases can indeed trigger SUð2Þ L breaking even in the presence of positive μ 2 H . To ensure the new heavy fermion masses will not contradict our assumption that D is the DM candidate, we will take them to be 1 TeV heavier than D. However, our results are not sensitive to the precise values of the new heavy fermion masses as long as they are much heavier than the SM Higgs mass.
In the left columns of Fig. 3, from top to bottom, we project out the allowed regions of λ H , λ Φ , λ HΔ and λ HΦ on the ðλ H ; λ HΦ Þ, ðλ H ; λ HΔ Þ and ðλ Φ ; λ HΦ Þ planes respectively. The HP constraints are shown in orange while the combined (VS þ PU þ HP) constraints are shown in magenta. We notice that the allowed regions for Higgs physics constraints are symmetric under λ HΦ;HΔ → −λ HΦ;HΔ in these three plots. That is because the Higgs mass m h 1 and the mixing with other scalars are invariant under λ HΦ → −λ HΦ and/or λ HΔ → −λ HΔ with M HΔ → −M HΔ as can be seen from Eq. (17). For values of λ H close to the SM one m 2 h =2v 2 ¼ 0.13, the 125 GeV Higgs observed at the LHC is very much SM-like, and one expects the mixing effects controlled by λ HΦ;HΔ should be minuscule. This fact is clearly reflected in the top and middle plots that λ HΦ and λ HΔ are confined to be small when λ H → 0.13.
The Higgs data constraints alone sets a lower limit for λ H ≥ 0.13 but not for λ Φ . The lower limit stems from the fact that in the limit of v ≪ v Φ and v Δ , the mixing among h, ϕ 2 and δ 3 can only decrease the lightest eigenstate mass m h 1 10 which can be only compensated by a larger λ H (≥ 0.13). To increase m h 1 , one would need λ H ≥ 0.13. By including the constraints from VS and PU conditions, one obtains 0.13 ≤ λ H ≤ 4.0 and 0 ≤ λ Φ ≤ 4.19. The allowed range of λ HΔ is wider than that of λ HΦ is simply because the value of v Φ is fixed but v Δ is varied in our setup. In general the theoretical (VS þ PU) constraints (in particular the PU constraints) are much stringent than the Higgs data constraint. Similarly, in the right column of Fig. 3, we plot the allowed regions from top to bottom for the ðλ 0 H ; λ HΦ Þ, ðλ 0 H ; λ HΔ Þ and ðλ Δ ; λ HΔ Þ planes. Since as indicated in the left panels current LHC Higgs data is more sensitive to the parameters λ H , λ HΦ and to a lesser extent λ HΔ but not others, we will not show the Higgs data constraints in all the subsequent plots. Instead we present only the theoretical (VS þ PU) constraints in red and the combined (VS þ PU þ HP) constraints in magenta. It is clear that FIG. 5. Comparison between signal strengths for the τ þ τ − and γγ. The scatter plot represents the prediction from G2HDM and the error bar represents the 1σ region for the current measured signal strengths. 10 Mixing effects that lead to decreasing the lightest eigenstate mass can also occur in the fermion case. One famous example is the Type-I seesaw mechanism, in which the contribution from the heavy right-handed neutrino to the light neutrino mass is actually negative which can be made positive via field redefinition. combining the Higgs data constraints with the theoretical constraints can further reduce the allowed regions for these parameters-most significantly on the ðλ 0 H ; λ HΦ Þ plane. Allowed regions of other two-dimensional planes of diagonal versus off-diagonal parameters will be summarized in the next section.
Next, we present the impacts of HP constraints on some of purely off-diagonal parameter planes ðλ ij ; λ kl Þ in Fig. 4. Comparing the two parameters λ HΦ and λ 0 HΦ , their favored regions by the Higgs data behave quite differently as they are constrained in different ways. The exclusion of the region λ HΦ ≳ 4 is owing to the SM Higgs mass, whereas λ 0 HΦ is forced to be greater than −1, in light of prohibition of tachyonic modes for any of the scalars. Overall, λ HΦ is more constrained by the Higgs physics data than λ ΦΔ , λ HΔ and λ 0 HΦ . Allowed regions on the other planes of off-diagonal versus off-diagonal parameters will be summarized in the next section.
In addition to the above bounds on λ s, we find that the mass parameter M ΦΔ has a lower limit ∼ − 0.56 GeV while the mass parameter M HΔ and the VEV v Δ remain unconstrained within their ranges of scanning. The lower bound on M ΦΔ arises from the observation that the B term defined in Eq. (21) must be negative in order to attain positive M D and MΔ. Note that since v Δ and v Φ are much larger than v, B can then be approximated as −2ð4v Δ m 2  [39]. The light and dark gray bands indicate the 2σ and 1σ errors in μ γγ ggH respectively. Below, the relative (lower-left) and absolute (lower-right) contribution to μ γγ ggH from new physics and its correlation with the mass of the charged Higgs.
Before leaving this section, we discuss the impact of the measured Higgs diphoton signal strength μ γγ ggH from the LHC. Note that a similar analysis was presented in the previous work [13]. We here improve our previous analysis by including effects of the new heavy fermion loops on both the Higgs production and the decay into two photons. In addition, we have also checked the Higgs to τ þ τ − decay channel which has a larger uncertainty (a factor of 2 or so) than the diphoton channel [42], as shown in Fig. 5. Note that μ ττ ggH is equal to Γðh 1 → ggÞ=Γ SM ðh → ggÞ, since the h 1 → τ þ τ − decay width differs from the SM one by O 2 11 cancelling the total decay width ratio Γ SM h =Γ h 1 ≈ O −2 11 . One can expect a similar behavior in other decay channels such as bb, W þ W − and ZZ, which also have bigger uncertainties. We would like to come back to this point for a more complete analysis in the future, when more data are collected at the LHC.
In Fig. 6, we show the scan results of μ ggH versus O 2 11 (top-left panel) and O 21 =O 11 (top-right panel), as well as the relative (bottom-left panel) and absolute (bottom-right panel) contributions from new physics to μ ggH . The relative contribution refers to the contribution from either the charged Higgs or heavy fermions to μ ggh divided by the two contributions combined, while the absolute contribution denotes the individual amplitude in Eq. (65) without the common coefficient in front. The Higgs constraints alone are labeled by orange dots, while the combined (VS þ PU þ HP) constraints are labeled by magenta crosses. Our results indicate that our model has no problem having μ ggH around the experimental central value of 0.81 with the 1σ and 2σ errors marked by the dark and light gray regions respectively. One can see that there are also points accumulating near the SM value μ γγ ggH ¼ 1, which is barely inside the 1σ range. In the top-left panel of Fig. 6, for O 2 11 < 0.75 we can see that the points allowed by all constraints (magenta cross) split into three branches for small O 2 11 , depending on values of O 21 =O 11 . The upper, middle and lower branches correspond to O 21 =O 11 > 0.5, −0.5 < O 21 =O 11 < 0.5 and O 21 =O 11 < −0.5, respectively, as can be visually identified from the top-right panel of Fig. 6. While this ratio of O 21 =O 11 appears several places in our μ γγ ggH calculation, this shape is mostly controlled by the heavy colored fermion contributions to the partial decay width of h 1 decays into two gluons in Eq. (69). This two gluon partial width is multiplied by the diphoton partial width, which also contains some less significant effects of this ratio through the heavy charged fermions as well as the charged Higgs contributions, communicating the effects of this ratio to the whole signal strength μ γγ ggH in Eq. (64). The deviation from the SM predicted μ γγ ggH ¼ 1 has three sources: the overall factor O 2 11 in Eqs. (65) and (69), and the extra loop contributions from the new heavy fermions and charged Higgs. Figure 6 shows that the combined effects from these three factors tend to decrease the signal strength: μ γγ ggH ≲ 1. In the end, any enhancement is not strong enough to make μ γγ ggH > 1 as it has to overcome the overall factor O 2 11 . An important reason for μ γγ ggH ≲ 1 is that the new heavy fermion contributions (in additional to the top quark loop) cancel the dominant contribution of the SM W AE loop for h 1 → γγ.
Furthermore, it might seem strange to find that values of μ γγ ggH do not fall sharply to the SM value of 1 as O 2 11 grows. However, upon closer inspection of Eq. (65), we notice that the charged Higgs, whether it is heavy or light, will still contribute to the diphoton channel even if O 11 ¼ 1. This is because O 21 and O 31 will be vanishing and C h in Eq. (66) becomes unity. A relatively light charged Higgs mass can indeed move μ γγ ggH away from the SM value even if the mixing is small, i.e., O 2 11 ≈ 1. When all the constraints are taken into account, the contribution from the charged Higgs dominates over heavy fermion ones completely for m H AE ≲ 400 GeV, but they become comparable around m H AE ≳ 10 3 GeV, as demonstrated by the lower-right panel of Fig. 6. It is important to note that while the lower panels of Fig. 6 apparently show an upper bound for the mass of the charged Higgs, this is actually the maximum value reached with our choice of scan range of v Δ and has no theoretical implication on the upper limit of the charged Higgs mass.
To further illustrate the role of new physics contributions from the charged Higgs and heavy fermions in G2HDM, we compare the new contributions with the SM contributions in the loops of Γðh 1 → γγÞ in Fig. 7. At the top-left panel the ratio of charged Higgs loop to SM loops is shown, while at the top-right panel we have the ratio of total contributions from new physics loops (charged Higgs plus new heavy fermions) to SM loops. At the two bottom panels we show the ratio of new heavy fermions to SM loops projected on the common heavy fermion mass (left) and charged Higgs mass (right).
Lastly, with the final allowed parameter space determined, we compute the mean values and the standard deviations for all the scalar masses using the allowed points. We thus obtain distributions for the masses of all the scalars in G2HDM. The resulting mass spreads are shown in Fig. 8, with dots indicating their mean values and error bars for the AE1σ deviation. One can easily see that while the DM candidate D in G2HDM has a mass around 1-3 TeV, all other scalars are much heavier (≳10 TeV). Note that, while we have constrained all the quartic couplings in the scalar potential, the cubic couplings M HΔ and M ΦΔ remain untamed in a very wide range. By choosing suitable ranges for these parameters, e.g., by tuning them to achieve 4AC ≪ B 2 in Eq. (20), one can give D a much lighter mass.

V. SUMMARY
G2HDM was proposed to promote the discrete Z 2 symmetry, that ensures the stability of DM in IHDM, to a local gauge symmetry by embedding the two Higgs doublets into the fundamental representation of the new SUð2Þ H gauge group. In this work we have studied the theoretical constraints from vacuum stability and perturbative unitarity imposed on the scalar potential of G2HDM at tree level. The 125 GeV Higgs physics data from the LHC are also taken into account, in particular the Higgs mass and its diphoton signal strength. We have improved the G2HDM scalar potential by including two new couplings λ 0 H and λ 0 HΦ , which were missing in our previous work. Although these two new couplings do not alter the minimization conditions of the scalar potential, their impacts on the scalar mass spectrum and effects on the VS, PU and HP constraints are analyzed in detail. We have recomputed the diphoton signal strength for the 125 GeV Higgs. We have included the contributions of the new heavy fermions in G2HDM and demonstrated that their effects can be significant in the diphoton channel. The charged Higgs contribution is also found to be significant if its mass is in the range of 100 to 300 GeV. Overall the diphoton signal strength is found to be ≲1 in G2HDM. In particular G2HDM can naturally accommodate the current ATLAS central value of 0.81 for the diphoton signal strength from gluon-gluon fusion production. We note that the corresponding central value from CMS is 1.10 [43] which is not favorable in G2HDM with the present numerical setup in this work. However we are quoting the LHC Run II data from ATLAS and CMS in our analysis, while the combined results from both experiments are not available yet. In Fig. 9, we summarize the allowed regions of the multidimensional parameter space projected on all the two-dimensional planes comprised of the four diagonal couplings λ H;Φ;Δ , λ 0 H and the four off-diagonal couplings λ HΦ;HΔ;ΦΔ , λ 0 HΦ . The upper red triangular block corresponds to (VS þ PU) constraints, while the lower magenta triangular FIG. 9. A summary of the parameter space allowed by the theoretical and phenomenological constraints. The red regions show the results from the theoretical constraints (VS þ PU) of Sec. III. The magenta regions are constrained by Higgs physics as well as the theoretical constraints (HP þ VS þ PU), as discussed in Sec. IV. block corresponds to the (VS þ PU þ HP) constraints. The diagonal panels indicate the allowed ranges of the eight couplings. One can see that among the eight couplings only two of them λ H and λ HΦ are significantly constrained by (VS þ PU þ HP). Some of the couplings like λ 0 H , λ 0 HΦ and λ ΦΔ are loosely constrained as shown clearly in Fig. 9.
We emphasize that our analysis in this work is based upon tree-level VS and PU. One loop corrections to the scalar potential in G2HDM is important as it has been demonstrated in [44,45] that such loop corrections may lead to larger parameter space for perturbativity constraints in BSM. Perturbative unitarity constraints can also be improved by including the cubic scalar couplings M HΔ and M ΦΔ . Recently, cubic scalar couplings are found to be significantly improved by the unitarity constraints in various models beyond the SM if the center-of-mass energy is not taken asymptotically infinite [46][47][48]. Further improvement of our current analysis can be carried out by including renormalization group running effects for all the quartic couplings as well as the two cubic couplings. These issues are interesting but beyond the scope of this work. We would like to come back to these issues in the future.
G2HDM is an interesting variant of the popular IHDM with a neutral component in the second Higgs doublet as a DM candidate. It would be interesting to include DM constraints from cosmology, direct and indirect detections as well as collider searches in our analysis. This work is in progress [49] and we will report it elsewhere.