Broad composite resonances and their signals at the LHC

We explore the possibility that the left-handed third generation quark doublet $q_L$ is fully composite in composite Higgs models. The signature of the model is the presence of broad spin-1 resonances strongly interacting with the third generation quark doublet $q_L$. In this case, The $t\bar{t}$ resonance search channel is comparable in sensitivity to the di-lepton channel. In addition, the same-sign di-lepton channel in the $t\bar{t}\rho$ associate production can probe the large $g_\rho$ parameter space and complementary to the Drell-Yan production channels.

Introduction.-The most important signal of composite Higgs models is the presence of composite resonances. A spin-1 resonance similar to the ρ of QCD (denoted also as ρ here) is probably one of the most obvious targets for collider searches. A generic expectation is that composite resonances should be broad, due to its large coupling. However, in the Minimal Composite Higgs Model, the composite ρ-resonances tend to be narrow if they can only decay into the Standard Model (SM) particles. In this article, we clarify that it is an accidental feature of the minimal models, rather than a generic one. As a demonstration, we propose a concrete composite Higgs model in which the ρ-resonances are broad.
The symmetry structure of a composite Higgs model is a coset G/H, where the strong dynamics preserves H even after confinement. Hence, the composite resonances should fill complete multiplets of H. The SM gauge groups are embedded in H. In popular benchmarks of the composite Higgs model, the SM fermions are treated as elementary. The third generation fermions acquire their masses through mixing with composite fermions [1][2][3]. In this scenario, the coupling between the SM fermions and the ρ is suppressed by either ρ-SM gauge boson mixing or elementary-composite fermion mixing. The ρ-resonances couple strongly to other composite states, such as the longitudinal modes of the W and Z, and the Higgs boson. However, the lack of color factor enhancement and some accidental small factor lead to the decay widthmass ratio of the ρ resonance to be O(g 2 ρ /96π). Hence, the ρ-resonance appears to be narrow even if g ρ is sizable.
There are other ways of embedding the third generation fermions. In particular, they can be fully composite and part of a complete multiplet of H. In the previous literature, partly motivated by the large mass of the top quark, t R has been treated as fully composite singlet of H [4,5]. Though simple, this is not the only possibility. Heaviness of the third generation fermions motivates considering more of them potential candidates of composite fermions. We will explore this direction of model space. This gives very different predictions for the width of the ρ-resonances, which lead to qualitatively new fea-tures and challenges for collider searches.
The Model.-We present here a model which realizes the new features discussed in the Introduction. We begin with the frequently used coset SO(5)/SO (4). We consider the left-handed third generation quarks as fully composite, embedded as a 4 of SO(4), where P is a 4 × 4 unitary matrix. Under the decomposition SO(4) × U (1) X → SU (2) L × U (1) Y , the quartet can be decomposed as bi-doublets, q L = (t L , b L ) T and q X L = (X L , T L ) T have the SM quantum number (3, 2) 1/6 and (3, 2) 7/6 , respectively. We will assume that the right-handed top quark t R is elementary. In addition, we introduce an elementary doublet q X R = (X R , T R ) T with SM quantum number (3, 2) 7/6 , which pairs up with q X L and becomes massive [6]. We will write them as incomplete 5 of SO(5), t 5 R = (0, 0, 0, 0, t R ) T

2/3
and q X5 The spin-1 composite resonances, the ρs, span the adjoint of SO(4) = SU (2) L × SU (2) R . We will mainly focus on 3 of SU (2) L , ρ a L , with a = 1, 2, 3. The relevant Lagrangian can be written down following the standard Callan-Coleman-Wess-Zumino procedure [7][8][9]: arXiv:1901.01674v1 [hep-ph] 7 Jan 2019 The Q i s are a set of higher-order operators [7]. The top quark mass M t , the top partner masses M T , M X are given by: There are various indirect constraints on the parameter space of the composite Higgs model.
The anomalous couplings.-The first set of constraints come from the modification of the Zb LbL , Zt LtL , W t LbL couplings. The composite fermion kinetic term has an accidental P LR parity symmetry, which exchanges [7,10]. This protects the Zb LbL coupling at tree level, avoiding dangerous deviations at the order of ξ ≡ v 2 /f 2 . This can be seen explicitly by noticing the Lagrangian in Eq. (3) contains two poten- , as shown in Eq. (3). However, the mass terms in Eq. (3) do not preserve the P LR , leading to corrections to the Zb LbL coupling at one loop level. We have checked that their constraints on the parameter space are weaker than the S, T -parameters [12,13]. The modifications to the W t LbL , Zt LtL couplings arise at tree level, given by δg W t L b L ∼ δg Zt L t L ∼ −ξ/4 [14]. The bound comes from electroweak precision test (EWPT) is given by |δg Zt L t L | 8% [14,15], which limits ξ 0.32. The newest measurement of Zt LtL coupling from ttZ associated production at the 36.1 fb −1 LHC limits |δg Zt L t L | 10% (95% C.L.) [16], corresponding to ξ 0.4.
Oblique parameters.-In our model, the strong dynamics preserves a SO(4) symmetry. Since SO(4) contains the custodial SU (2), there is no tree level contribution to the T -parameter. The S-parameter receives a tree level contribution from the mixing of ρ and SM gauge bosons. One loop contributions to the T and S come from heavy quarks, ρ-resonances, and the modified Higgs boson gauge boson couplings. Since our model in Eq. (3) is non-renormalizable, the loop contributions to S, T are in principle incalculable, thus in the following we regulate the divergence by a cutoff Λ = 4πf . For the contribution from the loop of ρs, we have used the results of [17][18][19]; while for the fermion loops, we calculated them using the general formulae in [20].
There can also be additional contribution to the Sparameter from higher order operators. An operator which is particularly relevant is where E a L µν is given by [26], and in the unitary gauge. It can contribute to the mixing between ρ and SM gauge bosons, and hence shift the Sparameter. According to the so-called partial UV completion assumption, α 2 1/g 2 ρ [7]. Therefore, the kinetic mixing between ρ µ and W µ induced by Q 2 is sub-leading. We often define β 2 = g 2 ρ α 2 , with β 2 being a O(1) parameter. There is no similar contribution to the T -parameter, if we assume custodial symmetry is preserved in the UV completion. In Fig. 1, we have plotted the 95% C.L. bound from S, T measurement on the M ρ − g ρ plane with a ρ = m ρ /(g ρ f ) = 1/2 with different values of β 2 , y 1R . We have used the limits on S and T in Ref. [27]: with strong correlation 92%. Note that the mass of the top partner is roughly given by y 1R f , small y 1R will lead to strong constraint on f . This is because the contribution to the S, T has terms of O(M 2 t /M 2 T ), as can been seen from the addendum. Meanwhile, y 1R explicitly breaks the custodial symmetry and larger y 1R will lead to stronger constraint. We find that y 1R ∼ 2 gives the weakest bound on our parameter space, as can be seen from the figure. Note that β 2 = 1/4 significantly relaxes bounds in the small M ρ region, as the tree level contributions to the S-parameter from the higher dimension operator Q 2 and mass term cancel. The bounds come from EWPT can be further relaxed if there is new positive contribution to the T -parameter [17,18,[28][29][30].
Collider signal.-The most significant difference between the collider signal of the spin-1 composite resonances in our model and those of the previously used benchmarks is in the width. The branching ratios into different final states and the total decay width for the neutral resonance ρ 0 are shown in Fig. 2. Since q L is fully composite, its coupling to the ρ is of the order g ρ . The dominant decay channels are tt, bb in the mass region M ρ < 2M X . In the mass region M ρ > 2M X , the decay into top partner pair is significant, which is almost half of the total decay widths in the large M ρ region. The branching ratio to the di-boson final state is suppressed by a factor of a 4 ρ /(2N c ). The suppression of the di-boson branching ratio, especially at small a ρ , makes them much less relevant. This is very different from the well-studied cases, where the di-boson channel is the most sensitive [31].
For broad resonances, the usual narrow width approximation does not apply. It is not correct to just add a large constant width to the propagator either. Instead, we need to replace the propagator as where √ŝ is the parton center of mass energy. This has a significant impact on the shape of the resonance at the LHC, as shown in Fig. 3.
There is no LHC search fully optimized for the broad resonances presented here. Achieving maximal sensitivity will be a challenge which deserves much more detailed studies. In the following, we will recast some of the LHC searches which still have sensitivity and highlight the difference with the well studied benchmarks. First of all, the searches in di-boson channel are not sensitive due to its suppressed branching ratio. At the same time, the limit set by searching for narrow resonances in the tt, bb, tb/tb and + − final states will not apply if Γ ρ /M ρ > 40%. The systematic uncertainties on the backgrounds will have a large impact for the large width case. There are several broad resonance searches at the LHC in the above channels, but most of the searches have used the constant decay width approximation which could mis-model the signal. For the tt channel, the large width effect has been considered up to Γ ρ /M ρ ∼ 30% by the ATLAS at 36.1 fb −1 [32] and the CMS at 35.9 fb −1 [33]. While the ATLAS measures only the semi-leptonic final state, the CMS analysis combines all possible final states and is more sensitive. In Fig. 4, we plot the current limits and the projected 3 ab −1 reach for the tt channel (red shaded region) based on the CMS result. The colored regions are truncated at g ρ ∼ 4 (Γ ρ /M ρ 30%), beyond which reliable extrapolations from current searches are not possible. When g ρ increases, the reach of M ρ first decreases because of the suppression of the coupling between the ρ resonance and valence quarks at large g ρ . It then increases as the bb initiated production becomes important. We have taken into account the difference between the dynamical width propagator in Eq. (8) and the constant decay width approximation. The possibility of a broad ρ 0 decaying into + − has been studied by ATLAS at 36.1 fb −1 [34], up to Γ ρ /M ρ = 32%. The corresponding limit and its extrapolation to 3 ab −1 are shown in blue regions in Fig. 4. The mass reach in low g ρ region is higher than through the tt channel. But in the high g ρ region, due to the branching ratio suppression, + − will be worse than the tt channel.
Currently, there is no strong constraint from the bb channel. ATLAS has searched for a broad bb resonance up to Γ ρ /M ρ = 15% in Ref. [35], but the constraint is 4: The current and projected constraints. The parameter benchmarks are y1R = 2, aρ = 1/2 and c1 = 1. The tt and + − bounds are based on the CMS [33] and ATLAS [34] measurements, respectively. When calculating the signal cross sections, we have taken the dynamical width effect into account. For the ttρ → tttt channel, we use the SSDL event contour N ( ± ± + jets) = 20 to set an estimate for the 3 ab −1 LHC.
too week to be shown in Fig. 4 due to its low integrated luminosity (3.2 fb −1 ). CMS considers both the narrow and broad resonances [36], in which the di-jet final state is searched. The study considers the dynamical width effect, and gives results for Γ ρ /M ρ up to 30%. Since no b-tagging is used in this search, the limit is weak. Besides the Drell-Yan processes, there are other channels sensitive to our scenario. Since the left-handed top is strongly coupled, the same-sign di-lepton (SSDL) channel in the four top final state pp → ttρ → tttt can be quite sensitive [37]. We expect that this channel has dependence to the modeling of the width. The expected sensitivity in our parameter space set by requiring 20 SSDL signal events (the green contour) is shown in Fig. 4. This channel can cover the large g ρ region, which is hard to probe via Drell-Yan process.
The signature in the large coupling region g ρ 4 would be very broad heavy resonance in the + − , tt, bb, tb/tb final states. One possible way to enhance the sensitivity of the searches for the broad resonance is to consider the interference between the signal and the SM irreducible background. This is similar to explore energy growing behavior from the higher dimension (four fermion) operators [38][39][40][41][42][43][44][45][46]. Since our ρ resonance is color neutral, the Drell-Yan production channel do not interfere with the QCD tt background. The t-channel bb → bb and the Drell-Yan + − do have interference with the SM irreducible backgrounds, respectively. However, due to the suppression of bottom PDF at high energy and the suppression of the di-lepton branching ratio at large g ρ , they don't have significant sensitivity to the region M ρ 4 TeV, g ρ 4. On the other hand, the productions of top partners T and X provide another probes of our model. Compare to the pair production process, the single production of T , X can reach a higher mass region. We estimate the sensitivity in the SSDL channel from singly produced of the lighter top partner, i.e. the charge-5/3 top partner X by the constant number of event N ( ± ± + jets) = 20. It turns out that such a method can reach M X ∼ 2.6 TeV (corresponding to ξ ∼ 0.036) in the parameter choice y 1R = 2, a ρ = 1/2, c 1 = 1. There can be additional handles on the signal, which could become important while reconstruction of a sharp resonance is less effective. For example, the ρ resonance strongly interacts with the left-handed top quarks, the the polarization measurement of the top quarks in the tt final states can also help improve the sensitivity.
Conclusion.-In this letter, we have studied the scenario that the left-handed third generation quark doublet q L = (t L , b L ) T are massless bound state of the strong dynamics, using the minimal coset SO(5)/SO(4) as an example. We have considered constraints on our model from the EWPT (S, T -parameters and δg Lb ) and direct searches at the LHC. Instead of in the di-boson final state in the case of narrow spin-1 resonances in the Minimal Composite Higgs Model, the smoking gun of the signature in our model is the broad resonances in the tt, bb, + − , tb/tb channels and four top final state. We have recast the searches at the present LHC and made projections at the HL-LHC. We find that tt is comparable to the di-lepton channel in our model and the SSDL channel in the four top final state can probe the large g ρ region. Further studies, taking into account additional information such as top angular distribution and polarization, are needed to fully optimize the search for such broad composite resonances. Supplementary Materials: analytical formulae for the S, T -parameters.-In this addendum, we list the analytical formulae for the S, T -parameters in our model. We assume that ξ is small and keep the leading terms in ξ expansion. As discussed in the main text, the total contribution can be divided into three classes: the fermion loop, the ρ resonance loop and the Higgs loop with modified Higgs gauge boson coupling: