Custodial symmetry violation in the Georgi-Machacek model

We study the effects of custodial symmetry violation in the Georgi-Machacek (GM) model. The GM model adds isospin-triplet scalars to the Standard Model in a way that preserves custodial symmetry at tree level; however, this custodial symmetry has long been known to be violated at the one-loop level by hypercharge interactions. We consider the custodial-symmetric GM model to arise at some high scale as a result of an unspecified ultraviolet completion, and quantify the custodial symmetry violation induced as the model is run down to the weak scale. The measured value of the eletroweak rho parameter (along with unitarity considerations) lets us constrain the scale of the ultraviolet completion to lie below tens to hundreds of TeV. Subject to this constraint, we quantify the size of other custodial-symmetry-violating effects at the weak scale, including custodial symmetry violation in the couplings of the 125 GeV Higgs boson to W and Z boson pairs and mixings and mass splittings among the additional Higgs bosons in the theory. We find that these effects are small enough that they are unlikely to be probed by the Large Hadron Collider, but may be detectable at a future e+e- collider.

We study the effects of custodial symmetry violation in the Georgi-Machacek (GM) model. The GM model adds isospin-triplet scalars to the Standard Model in a way that preserves custodial symmetry at tree level; however, this custodial symmetry has long been known to be violated at the one-loop level by hypercharge interactions. We consider the custodial-symmetric GM model to arise at some high scale as a result of an unspecified ultraviolet completion, and quantify the custodial symmetry violation induced as the model is run down to the weak scale. The measured value of the eletroweak ρ parameter (along with unitarity considerations) lets us constrain the scale of the ultraviolet completion to lie below tens to hundreds of TeV. Subject to this constraint, we quantify the size of other custodial-symmetry-violating effects at the weak scale, including custodial symmetry violation in the couplings of the 125 GeV Higgs boson to W and Z boson pairs and mixings and mass splittings among the additional Higgs bosons in the theory. We find that these effects are small enough that they are unlikely to be probed by the Large Hadron Collider, but may be detectable at a future e + e − collider.

I. INTRODUCTION
With the discovery of a Standard Model (SM)-like Higgs boson at the CERN Large Hadron Collider (LHC) in 2012 [1], we have the first direct access to the dynamics of electroweak symmetry breaking. The simplest implementation of this dynamics is through a single complex scalar field transforming as a doublet under the weak SU(2) L gauge symmetry; this is consistent with experimental data to date [2].
While at least one SU(2) L doublet is required to generate the masses of the SM fermions in a gauge-invariant way, the masses of the W and Z bosons can in principle also receive contributions from scalars in larger representations of SU(2) L . Such an extension to the Higgs sector is severely constrained by measurements of the ρ parameter [3], defined as the ratio of the strengths of the neutral and charged weak currents in the low-energy limit and measured to very high precision via the global electroweak fit [4]. Indeed, unless the vacuum expectation values (vevs) of the larger representations are negligibly small, the only viable models are those that preserve ρ = 1 at tree level: i. models with extra SU(2) L doublet(s) and/or singlet(s); ii. a model with an extra SU(2) L septet with appropriately-chosen hypercharge [5,6]; and iii. the Georgi-Machacek (GM) model [7,8] and its generalizations to larger SU(2) L representations [9][10][11][12][13].
In this paper we consider the GM model. In addition to the usual SU(2) L doublet, this model contains two SU(2) Ltriplet scalar fields, arranged in such a way that the scalar potential is invariant under a global SU(2) L ×SU(2) R symmetry; upon electroweak symmetry breaking, this global symmetry breaks down to its diagonal subgroup [known as the custodial SU (2)] and ρ = 1 is thereby preserved. The GM model gives rise to a rich and exotic phenomenology, including singly-and doubly-charged scalars that couple to vector boson pairs at tree level and the possibility that the SM-like Higgs boson's couplings to W W and ZZ could be larger than in the SM. It has been used as a benchmark by the LHC experiments for interpreting searches for singly-charged Higgs bosons decaying into vector boson pairs [14,15].
However, it has been known since the early '90s that the custodial symmetry in the GM model holds only at tree level [16]: the global SU(2) R symmetry is explicitly violated by the gauging of hypercharge, which leads to an uncontrolled violation of the custodial symmetry at one loop. The most obvious manifestation of this is that the standard calculation of the Peskin-Takeuchi T parameter [17] yields an infinite result; this infinity is to be cancelled by a counterterm that is absent in the SU(2) L ×SU(2) R -invariant potential of the GM model but appears in the full gauge-invariant but custodial-symmetry-violating theory [16].
A further manifestation, most relevant for our purposes, is that it is not possible to compute a consistent set of renormalization group equations (RGEs) for the Lagrangian parameters of the custodial-symmetric GM model unless one sets the hypercharge gauge coupling to zero [18]. Instead, we use the RGEs for the full gauge-invariant but custodial-symmetry-violating potential. These imply that it is possible to choose the Lagrangian parameters to preserve the custodial symmetry, but only at one energy scale. As one moves away from that special scale, custodial symmetry violation builds up due to the renormalization group running. Reference [18] studied this effect by assuming that the theory is custodial-symmetric at the weak scale and quantifying the amount of custodial symmetry violation that develops as one runs to higher scales.
In this paper we take a different approach. We imagine that the custodial-symmetric GM model arises at some high scale, for example as a theory of composite scalars with an accidental global SU(2) L ×SU(2) R symmetry in the scalar sector. (Such models have been constructed in the context of little Higgs theories in Refs. [19,20].) Below the compositeness scale, custodial symmetry violation accumulates through the running of the Lagrangian parameters down to the weak scale. Weak-scale measurements of the ρ parameter can then be used to constrain how high the custodial-symmetric scale can be. Subject to this constraint, we can also quantify the physical effects of custodial symmetry violation in Higgs-sector observables, such as the ratio of the SM-like Higgs boson couplings to W W and ZZ and custodial-violating mixings and mass splittings among the additional scalars in the GM model. Working within a particular benchmark scenario for concreteness, we will show that the custodial-symmetric scale can be as high as tens to hundreds of TeV, and that the effects of custodial symmetry violation at the weak scale are typically too small to be detected at the LHC. The custodial-violation-induced mass splittings may however be detectable at a future e + e − collider. The fermiophobic scalars of the GM model acquire small fermion couplings due to custodialviolation-induced mixing, but the resulting branching ratios remain below the percent level in the benchmark that we study. We leave to future work a careful study of the fermionic decays of the would-be fermiophobic scalars for masses below about 160 GeV, where fermionic decays could compete against the loop-induced diphoton decays that otherwise put strong experimental constraints on such light scalars. This paper is organized as follows. In Sec. II we review the GM model with exact custodial symmetry in order to set our notation. In Sec. III we write down the most general gauge invariant scalar potential for the custodial-violating theory with the same field content. In Sec. IV we compute the masses and mixing angles of the physical scalars in the custodial-violating theory and derive formulas for the most interesting custodial-violating couplings. In Sec. V we collect the one-loop RGEs for the custodial-violating theory. In Sec. VI we describe our calculational procedure and give our numerical results. In Sec. VII we conclude. In Appendix A we give a translation between our notation and that of Ref. [18], and in Appendix B we give some details of our calculation of the RGEs.

II. GEORGI-MACHACEK MODEL WITH EXACT CUSTODIAL SYMMETRY
The scalar sector of the GM model [7,8] consists of the usual complex doublet (φ + , φ 0 ) with hypercharge 1 Y = 1, a real triplet (ξ + , ξ 0 , ξ − ) with Y = 0, and a complex triplet (χ ++ , χ + , χ 0 ) with Y = 2. The doublet is responsible for the fermion masses as in the SM. In order to make the global SU(2) L ×SU(2) R symmetry explicit, we write the doublet in the form of a bidoublet Φ and combine the triplets to form a bitriplet X: The vevs are defined by Φ = v φ √ 2 I 2×2 and X = v χ I 3×3 , where I is the appropriate identity matrix and the W and Z boson masses constrain Upon electroweak symmetry breaking, the global SU(2) L ×SU(2) R symmetry breaks down to the diagonal subgroup, which is the custodial SU(2) symmetry. The most general gauge-invariant scalar potential involving these fields that conserves custodial SU(2) is given, in the conventions of Ref. [21], by 2 Here the SU(2) generators for the doublet representation are τ a = σ a /2 with σ a being the Pauli matrices, the generators for the triplet representation are and the matrix U , which rotates X into the Cartesian basis, is given by [22] The minimization conditions for the scalar potential read The physical fields can be organized by their transformation properties under the custodial SU(2) symmetry into a 1 We use Q = T 3 + Y /2. 2 A translation table to other parameterizations in the literature has been given in the appendix of Ref. [21]. unitarity requires that the λ i obey the following relations [21,22]: Requiring that the scalar potential is bounded from below imposes the following constraints [21]: where and Eq. (17) must be satisfied for all values of ζ ∈ [ 1 3 , 1].

III. CUSTODIAL VIOLATION AND THE MOST GENERAL GAUGE-INVARIANT SCALAR POTENTIAL
In order to allow for custodial symmetry violation, we rewrite the scalar potential in Eq. (3) in the most general SU(2) L ×U(1) Y gauge invariant form, following Ref. [16]. We define the scalar fields in SU(2) L vector notation as with vevs given by [compare Eq. (9)], We use tildes to denote the vevs, parameters, and mass eigenstates of the custodial-violating theory. The vevs of these three fields will be determined by and will be constrained by the ρ parameter, For convenience, we define the conjugate multiplets, We also define the following matrix forms of the triplet fields, The most general gauge invariant scalar potential can then be written as Note thatλ 4 andM 1 are complex in general, while the rest of the parameters are real. We have adopted the same notation as in Eq. (3.2) of Ref. [16] for the coefficients of the quartic terms, and we have added the trilinear terms that were eliminated in Ref. [16] by the imposition of a Z 2 symmetry. This scalar potential has also been written down (for realλ 4 andM 1 ) in Ref. [18]; we give a translation table to their notation in Appendix A.
We note that the last term in Eq. (25) can also be written as where ijk is the totally antisymmetric tensor with 123 = +1.
In the custodially-symmetric limit, the Lagrangian parameters in the gauge-invariant scalar potential in Eq. (25) reduce to those in the custodially-symmetric potential in Eq. (3) according tõ in agreement with Ref. [16].
Replacing the fields with their vevs and assuming CP conservation, the most general scalar potential becomes Minimizing this potential yields three equations: When the SU(2) L ×SU(2) R symmetry is imposed, these conditions reduce to those in Eq. (6).

IV. PHYSICAL MASSES AND MIXING IN THE CUSTODIAL SYMMETRY VIOLATING THEORY
Isolating all terms quadratic in scalar fields from the potential and using Eqs. (29)(30)(31) to eliminateμ 2 2 ,μ 2 3 andμ 2 3 in favour of the vevs yields the following mass matrices for the physical scalars. There is only one doubly-charged scalar,H ++ 5 = χ ++ = H ++ 5 , and its mass is given by √ 2λ 4ṽχṽξ +M 1ṽξ +M 1ṽχ , We first transform this mass-squared matrix into the basis of custodial-symmetric states (H + 5 , H + 3 , G + ) using where the orthogonal matrix R + is defined according to with Because the custodial-symmetry-violating effects will be small, for practical purposes we can diagonalize the masssquared matrix M 2 + using first-order perturbation theory. To first order in the custodial violation, the masses of the singly-charged physical mass eigenstatesH + 5 andH + 3 are just given by the diagonal elements of the mass-squared matrix, The compositions of the mass eigenstates are given to first order using where M 2 is the mass-squared matrix in the appropriate basis. Applying this to the singly-charged states and using the fact that M 2 +,33 = 0, we get, We highlight the composition ofH + 5 in particular because the custodial symmetry violation results in an admixture of φ + into this state. This allowsH + 5 to couple to fermions, which does not occur in the custodial-symmetric GM model. Indeed, we can write the Feynman rule for theH + 5ū d vertex as where the coupling to fermions induced by the custodial symmetry violation is, to first order, For comparison, in the custodial-symmetric GM model we can write the analogous coupling of H + 3 to fermion pairs as κ Finally, there are three CP-even neutral scalars, whose mass-squared matrix in the basis (χ 0,r , ξ 0,r , φ 0,r ) is given by where We first transform this mass-squared matrix into the basis of custodial-symmetric states (H 0 5 , H 0 1 , φ 0,r ) using where the orthogonal matrix R r is defined according to with To first order in the custodial symmetry violation, the mass ofH 0 5 is given by It is most straightforward to find the masses ofh andH by diagonalizing the remaining 2 × 2 block of M 2 r as follows: The mixing angle that achieves this diagonalization is given by where the states are given in terms ofα by and we have defined cα = cosα, sα = sinα. (Note that these are not yet the mass eigenstates: there is still a small mixing with H 0 5 to be dealt with below.) We introduce a second orthogonal rotation matrix Rα, defined according to with The mass-squared matrix in the basis (H 0 5 , Hα, hα) is then given by Note that M 2 r,11 = M 2 r,11 . The masses ofh andH can then be written (to first order in the custodial symmetry violation) in terms of the diagonal elements of this matrix as We now use Eq. (44) to write the compositions of the CP-even neutral mass eigenstates to first order in the custodial violation asH We highlight the composition ofH 0 5 in particular because the custodial symmetry violation results in an admixture of φ 0,r into this state. This allowsH 0 5 to couple to fermions, which does not occur in the custodial-symmetric GM model. The coupling ofH 0 5 tof f , normalized to the corresponding coupling of the SM Higgs boson, is then given to first order in the custodial symmetry violation by Finally, the mixing of a small amount of custodial-fiveplet H 0 5 into the physical Higgs bosonh, together with v χ =ṽ ξ , leads to a violation of custodial symmetry in the couplings ofh to W W and ZZ. This is parameterized in terms of the physical observable where κh W and κh Z are the couplings ofh to W W and ZZ, respectively, normalized to the corresponding couplings of the SM Higgs boson. We can write this in terms of the vevs and the mixing with H 0 5 as follows: where the couplings of hα to W and Z boson pairs, including the effects ofṽ χ =ṽ ξ , are given bỹ the couplings of H 0 5 to W and Z boson pairs are given bỹ . (71)

V. RENORMALIZATION GROUP EQUATIONS FOR LAGRANGIAN PARAMETERS
In order to run the parameters down from a custodial-symmetric high scale to the weak scale, we need the RGEs. We determine these using the formalism presented in Ref. [23], some details of which are given in Appendix B. The resulting equations are then (with t ≡ log µ, where µ is the energy scale), where g 1 and g 2 are gauge couplings (see below) and y b , y t , and y τ are Yukawa couplings, normalized according to y f = √ 2m f /ṽ φ . These RGEs agree with those of Ref. [18] (for realλ 4 andM 1 ) after translating the notation for the Lagrangian parameters as in Appendix A. A few potential symmetries are apparent in these RGEs. Setting M 1 =M 1 =M 2 = 0, the potential becomes invariant under (χ, ξ) → (−χ, −ξ) and therefore these three parameters are not regenerated by the running. Setting insteadλ 4 =M 1 = 0, the potential becomes invariant under χ → −χ and therefore these two parameters are not regenerated by the running. Settingλ 4 =M 1 =M 2 = 0, the potential becomes invariant under ξ → −ξ and therefore these three parameters are not regenerated by the running. Finally, if all the Lagrangian parameters are taken to be real at some scale, as will be the case when the most general potential is matched onto the intrinsically CP-conserving custodial-symmetric Georgi-Machacek model, they remain real at all scales.
Throughout we use the GUT normalization g = 3 5 g 1 , g = g 2 , and g s = g 3 . The renormalization group equations for the electroweak gauge couplings, including all the particle content of the GM model in the spectrum, are [24], and that for the strong gauge coupling is the same as in the SM (including the top quark contribution), The RGEs for the Yukawa couplings are identical to those of the SM [25], In our numerical work we will ignore y b and y τ .
As a consistency check, we can turn off the custodial-violating parts of the RGEs by setting g 1 = 0 and substituting the relations given in Eq. (27). We then find a self-consistent set of RGEs for the custodial-preserving Lagrangian parameters: In this paper we imagine that the custodially-symmetric GM model emerges at some scale Λ as an effective theory of some unspecified ultraviolet (UV) completion. For example, the scalars in the GM model could be composites and the custodial symmetry an accidental global symmetry resulting from the particle content of the UV theory. The running of the scalar potential parameters down to the weak scale induces custodial symmetry violation. We can then use the experimental constraint on the ρ parameter at the weak scale to set an upper bound on the scale Λ. Subject to this constraint, we can also predict the size of other custodial symmetry violating effects such as mass splittings among the members of the custodial fiveplet and triplet scalars, mixing between scalars in different custodial-symmetry representations (which, for example, can induce fermionic decays of the otherwise fermiophobic H 5 states), and the value of the ratio λ W Z ≡ κ W /κ Z of the 125 GeV Higgs boson (predicted as λ W Z = 1 in custodial-symmetric theories).

Fixed Parameters Variable Parameters Dependent Parameters
For concreteness, we work within the context of the so-called H5plane benchmark, which is a two-dimensional slice through the custodial-symmetric GM model parameter space as defined in Table I at the weak scale. This benchmark was introduced in Ref. [26] for interpretation of LHC searches for H ± 5 and H ±± 5 , and its phenomenology was studied in some detail in Ref. [27]. The H5plane benchmark takes m 5 and s H as its two free parameters: this will allow us to plot our results as contours in the m 5 -s H plane. The benchmark is defined for m 5 values of 200 GeV and higher. We leave to future work a detailed study of the custodial violating effects at lower m 5 values.
We perform the calculations as follows. We start by specifying an input point in the custodial-symmetric GM model at the weak scale, using the H5plane benchmark. Because it is not possible to separate the scale of the GM model states from the SM weak scale so long as the triplets contribute to electroweak symmetry breaking, for the purposes of renormalization group running we will define the "weak scale" to be m 5 as defined in the custodial-symmetric low-scale input parameter set. We define the electroweak gauge couplings at the weak scale in terms of the inputs G F , M W , and M Z , and we take α s (M Z ) = 0.118 to define the strong coupling at the weak scale (we ignore the running of the strong coupling between M Z and m 5 ; this is a small effect because the strong coupling only enters in the running of the top Yukawa coupling). We extract the value of the top Yukawa coupling using the relation y t = √ 2m t /v φ evaluated in terms of the custodial-symmetric input parameters at the weak scale. For simplicity, we set y b = y τ = 0; their effects would be very small.
We then run the parameters of the custodial-symmetric scalar potential up to a scale Λ using the RGEs in Eqs. (72-87) but with g 1 set to zero. We also run the gauge couplings (including the actual value of g 1 ) and the top Yukawa coupling from m 5 to Λ using Eqs. (88-91). For the running we use fourth-order Runge-Kutta with a small step size. The result of this is a custodial-symmetric scalar potential at the scale Λ. At this stage we can check whether any of the quartic scalar couplings has grown large enough to violate perturbative unitarity (indicating that we have almost run into a Landau pole). This allows us to determine the maximum scale allowed by perturbativity. We also check whether the potential has become unbounded from below; this turns out not to happen in our scans of the H5plane benchmark. Because the potential is still custodial-symmetric, we can use the requirements for perturbative unitarity and boundedness-from-below as derived for the custodial-symmetric theory [21] as given at the end of Sec. II.
From the custodial-symmetric scalar potential at scale Λ, we then run back down to the scale m 5 using the full RGEs in Eqs. (72-91) with g 1 = 0. The nonzero hypercharge coupling induces custodial symmetry violation in the scalar potential, causing violation of the custodial-symmetry relations of Eq. (27) among the parameters of the most general gauge invariant scalar potential. Having determined the custodial violating parameters we can now solve the minimization conditions in Eqs. (29), (30), and (31) for the custodial-violating vevsṽ φ ,ṽ χ , andṽ ξ . First we solve Eq. (29) forṽ 2 φ in terms of the other vevs and plug this in to Eqs. (30) and (31), which we then solve numerically using a two-dimensional Newton's method. For the initial guess we takeṽ χ =ṽ ξ = v χ , where v χ is the custodial-symmetric triplet vev in our original weak-scale input point.
However, this procedure suffers from a complication. The definition of the original weak scale input point in the H5plane benchmark uses the measured m h and G F as input parameters. These are used to fix λ 1 and µ 2 2 in the weak-scale custodial-symmetric theory. After running the parameters up to the scale Λ using the custodial-symmetric RGEs (with g set to zero) and then running them back down to the weak scale with the full custodial violating RGEs, the new weak-scale calculations of mh and G F = 1/ √ 2(ṽ 2 φ + 4ṽ 2 χ + 4ṽ 2 ξ ) yield numbers that do not match the original input values. To address this, we need to adjust the custodially-symmetric weak-scale input values for λ 1 and µ 2 2 (while keeping all the other weak-scale inputs fixed) until we obtain the correct experimental values of mh and G F after implementing the custodial symmetry violation. We do this by defining two functions, , where λ 1 and µ 2 2 are the inputs at the weak scale, m calc h and G calc F are calculated using the procedure described above, and m expt h and G expt F are the desired (experimental) values. The solution is the point at which f 1 = f 2 = 0, which we find iteratively using a two-dimensional Newton's method. This involves running the full RGE machinery up and down multiple times and is the slowest part of our numerical work.
Having solved for the appropriate input values of λ 1 and µ 2 2 , we now have a self-consistent set of scalar potential input parameters at the weak scale (µ = m 5 ), corresponding to a custodial-symmetric theory at the high scale (µ = Λ), which we then run back down to obtain the custodial-violating theory at the weak scale (again m 5 ) with the correct predictions for m h and G F . We then calculate our desired observables including the ρ parameter, the mass splittings among the states of the would-be custodial multiplets, and the effects of the mixing among the would-be custodial eigenstates.
In the rest of this section we present our results as contour plots in the H5plane benchmark in the m 5 -s H plane. We emphasize that m 5 and s H here are defined as part of the weak-scale custodial-symmetric input parameter point, and do not directly correspond to the physical masses, couplings, or vevs of the corresponding parameter point in the weak-scale custodial-violating theory.

B. Constraints on the cutoff scale from perturbativity and the ρ parameter
We begin by determining the maximum scale allowed for the custodial-symmetric ultraviolet completion by running the custodial-symmetric model up until we hit a Landau pole. This is shown in the left panel of Fig. 1 in the H5plane benchmark. The shaded region at large s H in these plots is excluded by theoretical constraints on the custodialsymmetric model. We define the Landau pole as the scale at which any of the custodial-symmetric quartic couplings λ i becomes larger than 10 3 ; the true divergence happens extremely close to this scale. In the right panel of Fig. 1 we also show the scale at which the quartic couplings in the custodial-symmetric theory violate any of the conditions for perturbative unitarity of two-to-two scattering amplitudes given in Eq. (16). We can see that the scale at which perturbative unitarity is violated is roughly an order of magnitude below the scale of the Landau pole. Within the H5plane benchmark, if the theory is to remain perturbative the ultraviolet completion has to appear at 290 TeV or below, and the maximum scale of the Landau pole in this benchmark is around 2600 TeV. For m 5 400 GeV, the upper bound on s H from theory constraints in the H5plane benchmark is due to the perturbative unitarity constraint; therefore along this boundary the scale of perturbative unitarity violation is essentially the same as m 5 , and the Landau pole occurs around 10 TeV.
We also note that in the H5plane benchmark, the value of λ 2 at the weak scale grows linearly with m 5 (see Table I). This is responsible for the decrease in the scale of perturbative unitarity violation and the subsequent Landau pole with increasing m 5 at small s H values, and is a quirk of the H5plane benchmark.
In what follows, we take the scale of perturbative unitarity violation to be an upper bound on the scale of the custodial-symmetric theory, and we do not run above this scale.
The maximum allowed scale of the custodial-symmetric ultraviolet completion can also be constrained by the stringent experimental limits on the ρ parameter, as defined in Eq. (22). For this calculation (and all that follow), we bring to bear the full computational machinery described in the previous section, including adjusting the input values of λ 1 and µ 2 2 to obtain the correct measured values of G F and m h in the custodial-violating theory at the weak scale. We take the current value of ρ from the 2016 Particle Data Group electroweak fit [4], and require that the value of ρ in the weak-scale custodial-violating theory be within 2σ of this value; i.e., between ρ lower = 0.99991 and ρ upper = 1.00083. Because the deviation in the ρ parameter in the custodial-violating weak-scale theory grows as the scale of the custodial-symmetric ultraviolet completion increases, this constraint puts a stronger upper bound on the scale of the ultraviolet completion in part of the H5plane benchmark parameter space, as shown in the left panel of Fig. 2, where we also plot the upper bound from requiring perturbative unitarity. The ρ parameter constraint is stronger than that from perturbative unitarity for moderate s H values and m 5 below about 850 GeV.  Fig. 1 (solid lines), showing also the highest allowed custodial-symmetric scale after requiring that the ρ parameter remain within ±2σ of its experimental value [Eq. (103)] in the custodial-violating weak-scale theory (dashed lines). The range of scales allowed after imposing the ρ parameter constraint remains the same as in Fig. 1.
Right: the value of ρ in the weak-scale custodial-violating theory when the custodial-symmetric scale is taken as large as possible subject to perturbative unitarity at the high scale and the experimental limits on ρ. The values of ρ range between the ±2σ limits of 0.99991 and 1.00083.
In the right panel of Fig. 2 we plot contours of ρ at the weak scale in the custodial-violating theory after running down from the maximum scale allowed by the stronger of the perturbative unitarity and ρ parameter constraints. ρ > 1 in almost all of the H5plane benchmark, except for a tiny sliver of parameter space at low m 5 < 250 GeV and s H below 0.4.  (66)]. In what follows we maximize the custodialviolating effects by taking the scale of the custodial-symmetric theory as high as possible, subject to the constraints from perturbative unitarity and the ρ parameter.

C. Custodial violation in couplings
In Fig. 3 we plot the deviation of λh W Z from its SM value of 1 in the H5plane benchmark. The effect is tiny, reaching at most half a percent in a small region of the H5plane benchmark with m 5 250 GeV and moderate values of s H ; for larger m 5 , the deviation is below two per mille. This deviation is well below the sensitivity of the current experimental measurement at the LHC, λh W Z = 0.88 +0. 10 −0.09 [2]. It is also below the expected sensitivity obtained by combining the projections for the measurement precision of the SM Higgs couplings κ W and κ Z at the High-Luminosity LHC (a few percent) and the proposed International Linear e + e − Collider (ILC) (roughly half a percent) as summarized in Ref. [28]. The proposed Future Circular Collider (FCC-ee) could begin to reach the required precision, with projected sensitivity for κ W and κ Z of 1.5 to 2 per mille [29]. 4 In Figs. 4 and 5 we plot the custodial-violation-induced couplings and branching ratios ofH 0 5 andH ± 5 to fermions, respectively. TheH 0 5 coupling to fermions κH 0 5 f reaches a magnitude of at most 0.04 in the H5plane benchmark, leading to fermion-induced (e.g., gluon fusion) production cross sections at most (0.04) 2 = 1.6 × 10 −3 times that of a SM Higgs of the same mass. Potentially more interesting is the effect of this coupling on theH 0 5 decays: as shown in the right panel of Fig. 4, the branching ratio ofH 0 5 to fermions can reach almost half a percent in the H5plane benchmark. ForH 0 5 masses above 350 GeV, these fermionic decays are overwhelmingly into tt pairs. Similarly, theH ± 5 coupling to fermions κH f reaches a magnitude of at most 0.052 in the H5plane benchmark. Again, production processes involvingH + 5 coupling to fermions, such as associated production with a top quark, will have cross sections that are far too small to be interesting at the LHC. The branching ratio ofH + 5 → tb can reach 1.2%, as shown in the right panel of Fig. 5.
The custodial-violation-induced decays ofH 0 5 andH ± 5 to fermion pairs do not dramatically alter the phenomenology within the H5plane benchmark, which is defined only for m 5 ≥ 200 GeV. Potentially more interesting is the effect of fermionic decays of these particles for masses below the W W or W Z thresholds, when the dominant diboson decays of these scalars go off shell. In the custodial-symmetric GM model, H 0 5 decays to γγ and H + 5 decays to W + γ become interesting for these low masses [31,32]; competition from custodial-violation-induced fermionic decays could dramatically change the phenomenology in this mass region. We leave a detailed study to future work.

D. Custodial-violating mass splittings
Custodial symmetry violation also induces splittings between the masses of the otherwise-degenerate custodial fiveplet and triplet states. These splittings follow a universal pattern everywhere within the H5plane benchmark; we leave it to future work to determine whether this pattern holds in general scans of the entire GM model parameter space. We again maximize the custodial-violating effects in what follows by taking the scale of the custodial-symmetric theory as high as possible, subject to the constraints from perturbative unitarity and the ρ parameter.
Among the custodial-triplet mass eigenstates,H 0 3 is heavier thanH + 3 , and both of these masses are shifted up relative to the weak-scale custodial-symmetric input value of m 3 . The splittings are small, as shown in Fig. 6: the mass difference betweenH 0 3 andH + 3 reaches at most 5.3 GeV (left panel of Fig. 6). The shift of theH 0 3 mass upward from the input value of m 3 is shown in the right panel of Fig. 6, and is at most 9.1 GeV. The shift ofH + 3 from the input m 3 value is smaller, reaching at most 3.9 GeV.
Among the custodial-fiveplet mass eigenstates,H ++ 5 is the heaviest, followed byH + 5 and thenH 0 5 . Again the mass splittings are small, as shown in Fig. 7. The top left panel of Fig. 7 shows the mass difference betweenH ++ 5 and H 0 5 , which is at most 7.2 GeV. The mass ofH + 5 falls between these two, but closer to the lighterH 0 5 state: the mass difference betweenH + 5 andH 0 5 reaches at most 1.8 GeV, as shown in the top right panel of Fig. 7. The mass ofH 0 5 remains within 2.3 GeV of the weak-scale custodial-symmetric input value of m 5 , but can be heavier or lighter: this is plotted in the bottom left panel of Fig. 7. The mass ofH ++ 5 is always larger than m 5 , with the difference reaching a maximum of 9.0 GeV, as shown in the bottom right panel of Fig. 7. The smallness of these shifts of the physicalH 5 masses relative to the weak-scale custodial-symmetric input value of m 5 justifies our use of this input value on the x axis of the plots.
Finally, in Fig. 8 we plot the shift of the mass of the physical mass eigenstateH relative to the weak-scale custodialsymmetric input value of m H . TheH mass is shifted upwards over almost all of the H5plane benchmark, and the shift is by at most 5.6 GeV. We conclude that, within the H5plane benchmark and even allowing for custodial symmetry violation, the custodial-symmetric predictions for the masses of the scalars in the model are reliable to within better than 10 GeV.
Experimentally checking the mass degeneracy of the scalars within the custodial triplet and the custodial fiveplet has been proposed as a way to test the custodial symmetry in the GM model [33,34]. At the LHC, mass reconstruction of the H 3 states relies on their decays to dijets, H + 3 → cs, H 0 3 → bb [33]. Considering that the dijet invariant mass resolution at the LHC is not sufficient to kinematically separate the hadronic decays of the W and the Z with their 11 GeV mass difference, it will not be possible to resolve a custodial-symmetry-violation-induced mass splitting betweenH + 3 andH 0 3 of at most 5.3 GeV within the H5plane benchmark. Mass reconstruction of the H 5 states at the LHC relies on their decays to vector boson pairs V V . Reference [33] studied the fully-leptonic final states, in which the masses of H ++ 5 , H + 5 , and H 0 5 could be determined from the endpoint of the transverse mass distribution of the V V final state. The resolution is worse than for a dijet resonance. The ATLAS experiment has performed a search for H + 5 → W + Z → jj + − [14], in which reconstruction of a mass peak forH + 5 becomes possible; however, the mass resolution is still limited by the dijet invariant mass resolution of the LHC, which is too poor to resolve the custodial-symmetry-violation-induced mass splitting among theH 5 states spanning at most 7.2 GeV in the H5plane benchmark.
Prospects are somewhat better at the ILC, as studied in Ref. [34].H 0 5 andH ± 5 can be singly produced in e + e − collisions via vector boson fusion, or in association with a Z or W ∓ boson, respectively. In the clean lepton collider environment, the H 5 decays to dibosons can be reconstructed using the fully hadronic final states. With the ILC target dijet energy resolution of σ E = 0.3 × E jj GeV [35], the dijet resolution will be σ E 3 GeV for E jj 100 GeV, famously allowing for W and Z bosons to be distinguished in the all-hadronic channel. Unfortunately, even this excellent mass resolution is too poor to resolve the custodial-symmetry-violation-induced mass splitting betweenH + 5 andH 0 5 , which reaches at most 1.8 GeV in the H5plane benchmark. One could hope to do better by using the leptonic decays of H 0 5 → ZZ → 4 and H ± 5 → W ± Z → ± E miss T + − ; these suffer from smaller branching fractions, but may offer good enough mass resolution to detect the mass splitting effect of the custodial symmetry violation.

E. Direct search constraints
The most stringent direct search constraint on the custodial-symmetric H5plane benchmark comes from a CMS search for H ±± 5 produced in vector boson fusion and decaying to W ± W ± → ± ± E miss T [36]. This constraint excludes The fractional change inṽ χ relative to the weak-scale custodial-symmetric input v χ , defined asṽ χ vχ − 1. We work in the H5plane benchmark and take the scale of the custodial-symmetric theory to be as large as possible subject to perturbative unitarity and the ρ parameter constraint. The fractional change is always negative and its absolute value reaches a maximum of 1.0%.
s H above 0.25 for m 5 = 200 GeV, rising to s H = 0.55 at m 5 = 800 GeV [27]. We can apply this straightforwardly to the model with custodial symmetry violation by noting the following. First, as shown in the bottom right panel of Fig. 7, the physical mass ofH ++ 5 is at most 5 GeV higher than m 5 in the region of interest. Second, we show in Fig. 9 the shift inṽ χ , which controls theH ±± 5 W ∓ W ∓ coupling and hence the vector boson fusion production cross section, relative to the value of v χ in the weak-scale custodial-symmetric theory. This shift is negative and amounts to less than a percent, so that the cross section is suppressed by no more than 2% due to the custodial symmetry violation. Finally, the custodial-symmetry-violation-induced mass splitting betweenH ++ 5 andH + 5 is less than 5 GeV in the region of interest, too small for the cascade decayH ±± 5 → W ±H ± 5 to compete significantly with the dominant H ±± 5 → W ± W ± signal channel. Thus we conclude that this direct search constraint on the custodial symmetry violating parameter space studied in this paper will be almost identical to that in the custodial-symmetric H5plane benchmark. 5

VII. CONCLUSIONS
In this paper we studied the effects of custodial symmetry violation in the Georgi-Machacek model. We assumed that the exactly custodial-symmetric GM model emerges at some high scale Λ as an effective low energy theory of an unspecified ultraviolet completion, and then ran the model down to the weak scale, giving rise to custodial symmetry violation from hypercharge interactions at one loop. The amount and pattern of custodial symmetry violation at the weak scale is uniquely determined by the parameters of the high-scale custodial-symmetric theory and the value of the scale Λ.
Starting from the the most general gauge invariant scalar potential, we derived the minimization conditions for the vevs and expressions for the physical scalar mass eigenstates. These allow us to calculate the custodial symmetry violating couplings of the physicalH 0 5 andH + 5 states to fermions, as well as the parameter λh W Z ≡ κh W /κh Z for the 125 GeV Higgs boson. We rederived the RGEs for the parameters of the most general scalar potential including CP violation, and confirm the results of Ref. [18] in the CP-conserving limit. Our numerical implementation of these results was complicated by the need to adjust the custodial-symmetric inputs to obtain the correct values of the physical 125 GeV Higgs mass and G F in the custodial-violating theory.
Working for concreteness in the H5plane benchmark scenario, we determined the maximum allowed scale of the custodial-symmetric theory imposing perturbative unitarity of two-to-two scalar scattering amplitudes and the experimental constraint on the ρ parameter. This allowed us to quantify the maximum possible deviation of λh W Z from its SM value, as well as the branching ratios of the otherwise-fermiophobicH 0 5 andH ± 5 scalars into fermions and the mass splittings within the custodial triplet and fiveplet. We found that the scale of the custodial-symmetric theory could be as high as tens to hundreds of TeV, with an upper bound of 290 TeV in the H5plane benchmark. Subject to this upper bound, we showed that λh W Z can deviate from its SM value by at most two per mille, and that the mass splittings within the custodial triplet and the custodial fiveplet are in the range of 1-8 GeV. Both of these effects are too small to be probed at the LHC, but may be detectable at a future e + e − collider. We also showed that the fermionic branching ratios ofH 0 5 andH + 5 can reach of order 1% in the H5plane benchmark, which is defined for m 5 ≥ 200 GeV. These fermionic decays may be more interesting forH 5 masses below the W W and W Z thresholds, where they can compete directly with the loop-induced γγ and W γ decay modes; we leave a detailed study of this low mass region to future work.

(A1)
Appendix B: Calculating the renormalization group equations We calculate the one-loop renormalization group equations (RGEs) in this paper using the formalism of Cheng, Eichten, and Lee [23]. They considered a Lagrangian for nonabelian gauge fields A a µ , real scalar fields φ i , and fermionic fields ψ α of the form where the gauge field strength tensor and covariant derivatives are Here g is the gauge coupling, θ a ij and t a αβ are the generators of the gauge group acting on the scalar and fermion representations, respectively, and C abc are the structure constants of the gauge group. The fermion masses m 0 and Yukawa couplings h i are matrices in the space of fermions. The (quartic) scalar potential is given by The quartic scalar couplings f ijkl are defined to be symmetric under interchange of any pair of indices; after collecting terms in the scalar potential, they can be extracted using .
The trilinear couplings and quadratic mass-squared coefficients in Eq. (25) can be integrated into this formalism by inserting one or two factors of a nondynamical scalar field φ 0 that has no gauge or fermion couplings, e.g., The trilinear and quadratic coefficients can then be treated in the same way as the quartic coupling coefficients f ijkl , setting one or two of ijkl equal to 0.
The RGEs for the quartic scalar couplings are given by Eq. (2.8) of Ref. [23], with t = log µ where µ is the energy scale and Repeated indices are to be summed over. In this expression the first three terms come from one-loop diagrams with two quartic scalar vertices, the fourth term comes from diagrams in which an external leg is decorated with a gauge boson loop, the fifth term is a four-scalar coupling induced by a closed loop of gauge bosons, the sixth term comes from diagrams in which an external leg is decorated with a fermion loop, and the last term is a four-scalar coupling induced by a closed box of fermions (see Fig. 3 in Ref. [23]). The new symbols in Eq. (B8) are defined as [23]: with repeated gauge indices summed over, and The formalism in Ref. [23] assumes a single gauge group and a single representation containing all the scalars. This can be straightforwardly generalized to our theory in which the scalars transform under SU(2) L ×U(1) Y as a doublet and two triplets as follows. We first write out all the scalar fields in terms of their real components, using ϕ 1 = (φ 1 + iφ 2 )/ √ 2 for the complex scalars. The covariant derivative for the scalars can then be written as where g and g are now the SU(2) L and U(1) Y gauge couplings and θ a ij and Y ii /2 are the SU(2) L and U(1) Y generators written as big matrices in the space of the 13 real scalars φ i in our model (plus one nondynamical scalar field φ 0 ). Equation (B8) must then be modified slightly to take into account the two gauge groups: The new gauge terms are given as follows. The S 2 (S) term comes from diagrams in which a U(1) Y gauge boson loop decorates one of the external scalar legs. Using Eq. (B9) with θ a ij = (Y i /2)δ ij , this term is given for each ijkl by The S 2 (S) term comes from diagrams in which an SU(2) L gauge boson loop decorates one of the external scalar legs. It will have different values depending on the SU(2) L representation of the scalar on each leg. Using the SU(2) L generators for doublets and triplets, we obtain from Eq. (B9) for each leg Summing over the four legs then gives, for each ijkl, − 12g 2 S 2 (S) = −3g 2 [S 2 (S) i + S 2 (S) j + S 2 (S) k + S 2 (S) l ] = − 3 4 g 2 n 2 i + n 2 j + n 2 k + n 2 l − 4 , where n i = 2T i + 1 is the dimensionality of the SU(2) L representation of the ith leg.
The 3Ā ijkl term in Eq. (B13) yields terms in the RGEs of order g 4 , g 4 , and g 2 g 2 . The couplings that give rise to these terms are the quartic scalar-scalar-vector-vector vertices, which can be found by examining the anti-commutation relations among the generators of the relevant gauge groups. We derive the form ofĀ ijkl as follows. First, starting from Eq. (B10) we absorb the gauge coupling into the generators and definē where t a are the appropriate SU(2) L generators acting on the relevant subspaces of the scalars and I n×n is the unit matrix on the subspace of scalars with a common hypercharge. Then, To actually calculate this, we writeĀ ijkl = 4 a,b=1 α ab ij α ab kl + α ab ik α ab jl + α ab il α ab kj , where for a real scalar multiplet the gauge-covariant terms yield, and for a complex scalar multiplet they give, Note that α ab ij is symmetric under interchange of i and j; care must be taken with factors of two in extracting the α ab ij from terms involving two different real scalar fields.
Finally for the fermion contributions, it is most straightforward to separate the contributions into a sum of terms each involving only leptons, down-type quarks, or up-type quarks. In our model only the SU(2) L doublet couples to fermions, as in the SM, and we can write the Yukawa matrices in the fermion mass basis as The contribution from diagrams in which an external leg is decorated with a fermion loop is then given for each ijkl by where Υ m = Tr with N f c being the number of colors of fermion type f . The contribution from the fermion box diagram will be − 12H ijkl = −4 (δ ij δ kl + δ ik δ jl + δ il δ jk ) 1 # of permutations of (ijkl) This yields the RGEs for the coefficients f ijkl defined in Eq. (B5). To obtain the RGEs for the individual quartic couplingsλ i in Eq. (25), one can write the f ijkl as linear combinations of theλ i and solve the set of linear equations. The multiple redundant solutions for eachλ i can be used as a check of the algebraic implementation.