Discovering the $h\to Z \gamma$ Decay in $t \bar t$ Associated Production

We explore the prospects to discover the $h \to Z \gamma$ decay in $t\bar t$-associated production, featuring a signal-to-background ratio of ${\cal O}(1)$. Performing a detailed analysis of the semi-leptonic $t \bar t $-decay channel, we demonstrate that the production mode could lead to a $\sim5\,\sigma$ discovery at the high-luminosity LHC, while the effective $h Z \gamma$ coupling could be extracted with a $\sim15\,\%$ accuracy. Extending the analysis to potential future $pp$ colliders with 27 TeV and 100 TeV center-of-mass energies, we also show that the latter would allow precision measurements at the few percent level, rendering possible precise extractions of the spin and CP properties of the Higgs boson.


I. INTRODUCTION
The decay of the Higgs boson to a photon and a weak Z boson, h → Zγ, has not been discovered yet. Measuring it can not only provide a further consistency test of the Standard Model (SM) of particle physics, but also has the potential to unveil new physics (NP) that could be hidden in other observables [1][2][3][4][5][6][7][8][9][10]. Moreover, in principle it furnishes a promising channel to extract spin and parity properties of the Higgs boson.
The decay is challenging to access via production modes entertained so far, such as gg → h, which lead to an expected significance of 2 σ with 100 fb −1 at the 14 TeV LHC [11]. Refined projections by the ATLAS and CMS collaborations show that even at the end of the LHC programme, with 3 ab −1 , a 5 σ discovery will be challenging [12]. Latest experimental searches set an upper limit of 6.1 (to 11.4) times the SM value for σ(pp → h → Zγ) [13,14].
The h → + − γ channel also offers the possibility to independently measure the spin [11,15] and CP [8] properties of the Higgs, but the low signal to background ratio makes it difficult to extract angular correlations or asymmetries in the inclusive search. Here and in the following always denotes electrons and muons.
In this article, we entertain the channel pp → tth, h → Zγ → + − γ, which enhances the prospects to discover the h → Zγ decay and to measure the corresponding effective coupling. In fact, the tth production mode has recently been observed by ATLAS and CMS, inviting to use it for further studies [16][17][18]. It profits in particular from the large Yukawa coupling of the top quark, such that the radiation of a Higgs boson from a tt state leads only to a modest suppression of the cross section. This promises a significantly enlarged signal-tobackground ratio compared to other channels like gluon * florian.goertz@mpi-hd.mpg.de † eric.madge@uni-mainz.de ‡ pedro.schwaller@uni-mainz.de § valentin.tenorth@mpi-hd.mpg.de fusion, where one starts inevitably with a further loopsuppressed signal, thereby increasing the prospects to measure spin and CP. We will both study the expected significance for the channel under consideration at the high-luminosity LHC (HL-LHC) as well as examine potential constraints on the coefficient of the effective hZγ coupling. Finally, we will extend the analysis to include a future 27 TeV (HE-LHC) and a 100 TeV pp collider, like the FCC hh .

II. SETUP
We consider the SM, augmented with the D = 6 operators relevant for the decay h → Zγ to leading approximation 1 , where H is the scalar Higgs doublet, parametrised after electroweak symmetry breaking (EWSB) as Here, v denotes the vacuum expectation value (VEV) of the Higgs field H = 1/ √ 2(0, v) T , which triggers EWSB, h is the physical radial Higgs boson and ϕ 1,2,3 are the goldstone modes. This setup allows us to study deviations from the SM in a model independent way, under the assumption that there is a mass gap between the SM and the NP. After EWSB, the operators (1) generate in particular the Lagrangian term at the tree-level, contributing to the h → Zγ decay, with where c HW,HB,γ are the coefficients of the operators (1) in the effective D = 6 Lagrangian. Note that the direction (3) is not very constrained yet such that still significant NP effects can be present [5,[19][20][21].
For the following analysis we define the ratio of the decay width in the presence of the operators (1) and the SM decay width (see, e.g., [4]) where the second equality is valid for small c Zγ . We will eventually study the constraints that can be set on κ Zγ , and thus on the Wilson coefficient c Zγ , from the process under consideration.
For the signal to remain observable after selection cuts, the analysis will have to be as inclusive as possible. Electrons, muons, and photons are reconstructed with high efficiencies. On the other hand, tagging tt-associated production and including isolation requirements, taking into account the probability of overlapping with some of the top decay products, will reduce the number of events. For a first estimate, we thus assume a selection efficiency of (10 − 15) %, comparable to the experimental efficiency of the di-photon channel [24], which we will corroborate quantitatively in an explicit analysis in the next section. This would finally lead to about S = (20 − 30) signal events per experiment.
The main irreducible background is ttZ production with radiation of a photon from initial or final states. At the 14 TeV LHC, the NLO QCD cross section with p T,γ > 10 GeV and |η γ | < 4.0 is σ(pp → ttZγ) = 9.3 fb, about ten times larger than the signal, resulting in B 0 ≈ 1870.
Among the reducible backgrounds, we expect the dominant contribution from pp → ttZj production where one jet is misidentified as a photon. Experimentally this background can be estimated by loosening the photon identification, however we can not simulate this reliably. Eventually the best approach will be to float the background normalisation to fit the data in the side-bands below and above m h . For the purpose of the present work we will increase the background cross section by 50 % to obtain more realistic estimates for the sensitivity. Including this factor and multiplying with the selection efficiency above we arrive at (280 − 420) background events. Whether other backgrounds are relevant will depend on the tt decay channel and on the analysis, but we expect them to be sub-leading and have a smooth m γ invariant mass distribution.
Once the γ + − invariant mass is restricted to a 10 GeV window around the Higgs mass, the background is reduced by another factor of ∼ 15, see below, and we would obtain B = (20 − 30) ≈ S, resulting in a 4.5 σ−5.5 σ sensitivity from a simple cut and count analysis. This can further be improved by fitting the invariant mass distribution with signal plus background and background only hypotheses. The potential to observe the h → + − γ channel in a low background environment is our main motivation to perform this study. In the next section we provide a detailed simulation for the semileptonic tt channel, to better understand how realistic the above estimate is.

IV. ANALYSIS
In order to get a solid estimate of the expected sensitivity at the HL-LHC with √ s = 14 TeV and an integrated luminosity of 3 ab −1 we simulate the signal process pp → tth with MadGraph5 aMC@NLO [25,26] at next-to-leading order (NLO) in QCD using the PDF4LHC15 nlo 30 pdfas PDF set [27] provided through LHAPDF6 [28].
Our value for the tthproduction cross-section is in good agreement with the results of the LHCHXSWG, quoted above. For the partonshowering we use the MadGraph-build-in Pythia 8.2 [29], only allowing for the h → Zγ and Z → + − decays and rescaling the cross-section by the branching fractions B(h → Zγ) = 1.54 · 10 −3 and B(Z → + − ) = 0.067. A fast detector simulation is done with Delphes 3.4.2 [30] using the HL-LHC detector card.
We also simulate the irreducible background pp → ttZγ without contributions from Higgs decays, obtaining a cross section of approximately 9.3 fb for p T,γ > 10 GeV and |η γ | < 4.
We focus on semi-leptonic tt decays (t → bjj,t →b −ν , or vice versa) as those are best to handle for a cutbased analysis. Still all top decays are allowed in Pythia to account for example for the possibility of τ 's being mistagged as leptons and therefore contributing to the semi-leptonic channel.
The reconstruction requirements for electrons (muons) are p T > 15 (10) GeV, |η| < 2.47 (2.7) and for photons p T > 5 GeV, |η| < 2.37. Jets are reconstructed with Fast-Jet 3 [31] using the anti-k t algorithm [32] with R = 0.4 and are considered to have p T,j > 25 GeV and |η| < 2.5. In addition the following selection requirements have to be fulfilled 2 : • Exactly three leptons (electrons and muons) satisfying the reconstruction requirements • Three or more jets • p T,j > 30 GeV for the first three jets • Missing energy E / T > 20 GeV • At least one b-tagged jet • At least one photon with p T,γ > 15 GeV • Z-reconstruction: opposite sign same flavour lepton pair with 76 GeV < m < 106 GeV • Higgs-reconstruction: 120 GeV < m γ < 130 GeV The numerical results for the signal and background are shown in Table I. To reconstruct the Z-boson we require a opposite sign, same flavour (OSSF) lepton pair in the invariant mass range 76 GeV < m < 106 GeV in the final state, avoiding contamination from top-decays. If more than one lepton pair fulfils this requirement, the one closer to the Z-mass is chosen. This lepton pair together with the highest-p T photon is used to reconstruct the Higgs mass. The invariant mass distribution of the γ + − system (before applying the m γ cut) is shown in Fig. 1. The signal clearly peaks at m γ = m h = 125 GeV and we see that by cutting on a window of m h ±5 GeV we can obtain S/B 1. The signal and background selection efficiencies for the semi-leptonic channel now follow from Table I as N ≡ N final /(N initial B semi−lept. ), N = S, B with B semi−lept. = 0.288 [23], where we obtain S = 0.14 and B = 0.0097. 3 In order to arrive at our final result for the expected significance and the anticipated constraint on κ Zγ we assume the same efficiencies over all top-decay channels and, as discussed before, enhance B by 50% to account for reducible backgrounds, such as pp → ttZj. We thus finally arrive at a total S = 186 × S ≈ 25 and B = 1.5 × 1862 × B ≈ 27, including now realistic analysis cuts and taking into account losses due to overlapping final state particles in a detector simulation. This result agrees well with our first estimate above.
Considering the statistical error of ∆B = √ B ≈ 5, we thus expect to establish a signal from the total rate alone with a significance of ∼ 5 σ at a single experiment. We further note that the recent developments in topreconstruction using boosted decision trees, allowing to identify hadronic top-decays with a high efficiency, could enhance the sensitivity [16][17][18]. Therefore our results can be seen as a conservative estimate even if other background processes are underestimated. 2 Note that these cuts are mainly meant to select/specify our signal and suppress other backgrounds rather than to separate it from the simulated (irreducible) background.
Here the tth production cross section amounts to 2.9 pb for 27 TeV [35] and approximately 33 pb for 100 TeV center of mass energy [36], which were reproduced by our MadGraph simulations. The background of ttZγ production features 46 fb (670 fb) at 27 TeV (100 TeV) with p T,γ > 10 GeV and |η γ | < 4. For simplicity and easier comparability we use a similar setting and the same reconstruction and selection requirements as for the HL-LHC. For the 100 TeV case we use the FCC hh Delphes Card.
Considering again the Z → + − channel, we obtain the cut-flows shown in Table II. The corresponding m γ invariant mass spectra can be found in the appendix (Fig. 3). For both scenarios the same extrapolation to include all top-decay channels and an enhancement of the background by 50 % as for the HL-LHC is performed. For the background, a cut of pT,γ > 10 GeV and |ηγ| < 4 is imposed at the generator level.

VI. CONSTRAINTS ON κZγ
In the following, we want to examine the expected constraints that can be set on κ Zγ from the process under consideration. To that end, we first calculate the predicted number of events N (κ Zγ ) = S(κ Zγ ) + B, where S(κ Zγ ) is obtained from the SM value S = 25 by multiplying with κ 2 Zγ , see (4). We further assume the SM to be true and calculate how many standard deviations ∆N (κ Zγ ) away the prediction N (κ Zγ ) is from N (κ Zγ = 1), which is the expected outcome of the experiment. The values of κ Zγ that lead to a discrepancy of more than n standard deviations are then expected to be excluded with a significance of n σ.
At envisaged future hadron colliders, a signal in this low background process could thus be established at a level of well beyond 5 σ, where the number of events clearly allows to pin down quantities like the spin of the Higgs boson or its CP properties and to perform precision tests of the effective hZγ coupling at the 1 % level.
At this level of precision, it becomes necessary to take into account potential systematic errors. On the theory side, the interpretation of the observed rate as a constraint on κ Zγ is affected by the uncertainty in σ(pp → tth), which is currently of order 10 % for the LHC. Anticipating some theory progress, in Fig. 2  Our projected sensitivities to κ Zγ are comparable to those in other Higgs production channels, which are on the order of 10 % (3 -4 %) at the HL-(HE-)LHC [35].
A further reduction of systematic errors could be achieved if one considers ratios of couplings such as κ Zγ /κ γγ in the tth channel. Such ratios are very sensitive to potential new physics patterns, for example additional charged fermions coupled to the Higgs have a stronger effect on κ γγ , since the contribution of the W boson loop strongly dominates the h → Zγ rate in the SM.

VII. CONCLUSIONS
We have explored the prospects to discover the decay of the Higgs boson to a photon and a Z boson in tt-associated Higgs production. Focusing our analysis on the semi-leptonic tt-decay channel, we demonstrated that the production mode considered could lead to a ∼ 5 σ discovery at the HL-LHC. Beyond that, we derived projected bounds on the effective hZγ coupling, κ Zγ , at the HL-LHC and future pp colliders with 27 TeV and 100 TeV center-of-mass energies, finding 1 σ constraints at the level of 15 %, 4 %, and 2 %, respectively. The sensitivity is comparable to or even exceeds that of future lepton colliders [37][38][39]. Finally, the corresponding S/B ratios of O(1) would also render possible precise extractions of the spin and CP properties of the Higgs boson. Furthermore in Fig. 4 we show the p-Value plots for κ Zγ for the 14, 27 and 100 TeV colliders, first without systematic errors and then including a 5 % systematic error.