Searches for heavy Higgs bosons in two-Higgs-doublet models and for t to ch decay using multilepton and diphoton final states in pp collisions at 8 TeV

Searches are presented for heavy scalar (H) and pseudoscalar (A) Higgs bosons posited in the two doublet model (2HDM) extensions of the standard model (SM). These searches are based on a data sample of pp collisions collected with the CMS experiment at the LHC at a center-of-mass energy of sqrt(s) = 8 TeV and corresponding to an integrated luminosity of 19.5 inverse femtobarns. The decays H to hh and A to Zh, where h denotes an SM-like Higgs boson, lead to events with three or more isolated charged leptons or with a photon pair accompanied by one or more isolated leptons. The search results are presented in terms of the H and A production cross sections times branching fractions and are further interpreted in terms of 2HDM parameters. We place 95% CL cross section upper limits of approximately 7 pb on sigma times B for H to hh and 2 pb for A to Zh. Also presented are the results of a search for the rare decay of the top quark that results in a charm quark and an SM Higgs boson, t to ch, the existence of which would indicate a nonzero flavor-changing Yukawa coupling of the top quark to the Higgs boson. We place a 95% CL upper limit of 0.56% on B(t to ch).


Introduction
The standard model (SM) has an outstanding record of consistency with experimental observations.It is not a complete theory, however, and since the recent discovery of a Higgs boson [1][2][3], attaining a better understanding of the mechanism responsible for electroweak symmetry breaking (EWSB) has become a central goal in particle physics.The experimental directions to pursue this goal include improved characterization of the Higgs boson properties, searches for new particles such as the members of an extended Higgs sector or the partners of the known elementary particles predicted by supersymmetric models, and searches for unusual processes such as rare decays of the top quark.Since the Higgs boson plays a critical role in EWSB, searches and studies of decays with the Higgs boson in the final state have become particularly attractive.
In many extensions of the SM, the Higgs sector includes two scalar doublets [4].The two Higgs doublet model (2HDM) [5] is a specific example of such an SM extension.In this model five physical Higgs sector particles survive EWSB: two neutral CP-even scalars (h, H), one neutral CP-odd pseudoscalar (A), and two charged scalars (H + , H − ) [6].For masses at or below the 1 TeV scale these particles can be produced at the LHC.Both the heavy scalar H and pseudoscalar A can decay into electroweak bosons, including the recently discovered Higgs boson.The branching fractions of H and A into final states containing one or more Higgs bosons h often dominate when kinematically accessible.For heavy scalars with masses below the top pair production threshold, the H → hh and A → Zh decays typically dominate over competing Yukawa-suppressed decays to bottom quarks, while for heavy scalars with masses above the top pair production threshold, these decays are often comparable in rate to decays into top pairs and are potentially more distinctive.
We describe a search for two members of the extended Higgs sector, H and A, via their decays H → hh and A → Zh, where h denotes the recently discovered SM-like Higgs boson [1][2][3].The final states used in this search consist of three or more charged leptons or a resonant photon pair accompanied by at least one charged lepton.(In the remainder of this paper, "lepton" refers to a charged lepton, e, µ, or hadronic decay of the τ-lepton, τ h ).The H → hh and A → Zh decays can yield multileptonic final states when h decays to WW * , ZZ * , or ττ.Similarly, the resonant decay h → γγ can provide a final state that contains a photon pair and one or more leptons from the decay of the other daughter particle.
Using the same dataset and technique, we also investigate the process t → ch, namely the flavor changing rare decay of the top quark to a Higgs boson accompanied by the charm quark in the tt → (bW)(ch) decay.The t → ch process can occur at an observable rate for some parameters of the 2HDM [7].Depending on how the h boson and t quark decay, both multilepton and diphoton+lepton final states can be produced.The CMS experiment has already published an upper limit on the t → ch branching fraction of 1.3% at 95% Confidence Level (CL) [8] with an inclusive multilepton search that uses the dataset analyzed here.We describe here a t → ch search using diphoton+lepton events and combine the results of the previously reported multilepton search with the present diphoton+lepton search.This combination results in a considerable improvement in the t → ch search sensitivity.
In this paper, we first briefly describe the CMS detector, data collection, and the detector simulation scheme in Section 2. We then describe in Section 3 the selection of events that are relevant for the search signatures followed by the event classification in Section 4, which calls for the data sample to be subdivided in a number of mutually exclusive channels based on the number and flavor of leptons, number of hadronically decaying τ leptons, photons, jet flavor, as well as the amount of missing transverse energy (E miss T ).A description of the SM background estimation in Section 5 precedes the channel-by-channel comparison of the observed number of events with the background estimation in Section 6.We next interpret in Section 7 these observations in terms of the standalone production and decay rates for H and A. Since these rates follow from the parameters of the 2HDM, we reexpress these results in terms of the appropriate 2HDM parameters.Finally, we selectively redeploy the H and A search procedure to detect the rare t → ch decay.
The multilepton component of this analysis closely follows the previously mentioned CMS inclusive multilepton analysis [8].In particular, the lepton reconstruction, SM background estimation procedures as well as the dataset used are identical in the two analyses and are therefore described minimally here.

Detector, data collection, and simulation
The central feature of the CMS detector is a superconducting solenoidal magnet of field strength 3.8 T. Within the field volume are a silicon pixel and strip tracker, a lead tungstate crystal calorimeter, and a brass-and-scintillator hadron calorimeter.The tracking detector covers the pseudorapidity region |η| < 2.5 and the calorimeters |η| < 3.0.Muon detectors based on gas-ionization detectors lie outside the solenoid, covering |η| < 2.4.A steel-and-quartz-fiber forward calorimeter extends the pseudorapidity coverage to |η| < 5.0.A detailed description of the detector as well as a description of the coordinate system and relevant kinematical variables can be found in Ref. [9].
The data sample used in this search corresponds to an integrated luminosity of 19.5 fb −1 recorded in 2012 with the CMS detector at the LHC.Dilepton triggers (dielectron, dimuon, muon-electron) and diphoton triggers are used for data collection.The transverse momentum (p T ) threshold for dilepton triggers is 17 GeV for the leading lepton and 8 GeV for the subleading lepton.Similarly, the p T thresholds for the diphoton trigger are 36 and 22 GeV.
The dominant SM backgrounds for this analysis such as tt quark pairs and diboson production are simulated using MADGRAPH (version 5.1.3.30)[10] generator.Quantum Chromodynamics (QCD) multijet events are generated with PYTHIA 8.1 [11].We use CTEQ6L1 leading-order parton distribution function (PDF) set [12].For the diboson + jets simulation, up to two jets are selected at the matrix element level in MADGRAPH.The detector simulation is performed with GEANT4 [13].The generation of signal events is performed using both the MADGRAPH and PYTHIA generators, with the description of detector response based on the CMS fast simulation program [14].

Particle reconstruction and preliminary event selection
The CMS experiment uses a particle-flow (PF) based event reconstruction [15,16], which takes into account information from all subdetectors, including charged-particle tracks from the tracking system and deposited energy from the electromagnetic and hadronic calorimeters.All particles in the event are classified into mutually exclusive types: electrons, muons, τ leptons, photons, charged hadrons, and neutral hadrons.
The electron and muon candidates used in this search are reconstructed from the tracker, calorimeter, and muon system measurements.Details of reconstruction and identification can be found in Ref. [17,18] for electrons and in Refs.[19,20] for muons.The electron and muon candidates are required to have p T ≥ 10 GeV and |η| < 2.4.For events triggered by the dilepton trigger, the leading electron or muon must have p T > 20 GeV in order to ensure maximal effi-ciency of the dilepton trigger.Hadronic decays of the τ lepton (τ h ) are reconstructed using the hadron-plus-strips (HPS) method [21] and must have the measured jet p T of the jet tagged as a τ h candidate to be greater than 20 GeV and |η| ≤ 2.3.
Photon candidates are reconstructed using the energy deposit clusters in the electromagnetic calorimeter [18,22].The candidate photons are required to satisfy shower shape requirements.In order to reject electrons misidentified as photons, the photon candidate must not match any of the tracks reconstructed with the pixel detector.Photon candidates are required to have p T ≥ 20 GeV and |η| < 2.5.For events triggered by the diphoton trigger, the leading (subleading) photon must have p T > 40 (25) GeV.
Jets are reconstructed by clustering PF particles using the anti-k T algorithm [23] with a distance parameter of 0.5 and are required to have |η| ≤ 2.5.Jets are further characterized as being "b-tagged" using the medium working point of the CMS Combined secondary-vertex (CSV) algorithm [24].They typically result from the decays of the b quark.The total hadronic transverse energy, H T , is the scalar sum of the p T of all jets with p T > 30 GeV.The E miss T in an event is defined to be the magnitude of the vectorial p T sum of all the PF candidates.The primary vertex for a candidate event is defined as the reconstructed collision vertex with the highest p 2 T sum of the associated tracks.It also must be within 24 cm from the center of the detector, along the beam axis (z direction), and within 2 cm in a direction transverse to the beam line [25].We require the candidate lepton to originate from within 0.5 cm in z of the primary vertex and that its impact parameter d xy between the track and the primary vertex in the plane transverse to the beam axis be at most 0.02 cm.
For electrons and muons, we define the relative isolation I rel of the candidate lepton to be the ratio of the p T sum of all other PF candidates that are reconstructed in a cone defined by ∆R = √ (∆η) 2 + (∆φ) 2 < 0.3 around the candidate to the p T of the candidate, and require I rel < 0.15.The photon isolation selection is similar, but varies as a function of the candidate p T and η.For the isolation of the τ h candidates, we require that the p T sum of all other particles in a cone of ∆R < 0.5 be less than 2 GeV.The isolation variable for leptons and photons is corrected for the contributions from pileup interactions [26].The combined efficiencies for trigger, reconstruction and identification are approximately 75% for electrons and 80% for muons.
A leptonically-decaying Z boson can lead to a trilepton event when the final-state radiation undergoes (internal or external) conversion and one of the leptons escapes detection.Therefore, we reject trilepton events with low missing transverse energy (E miss T < 30 GeV) when their three body invariant mass is consistent with the Z mass (i.e.m + − ± or m + − ± is between 75 and 105 GeV), even if m + − is not ( = e, µ).Finally, SM background from abundant low mass Drell-Yan production and low-mass resonances like J/ψ and Υ is suppressed by rejecting an event if it contains a dilepton pair with m + − below 12 GeV.

Event classification
We perform searches using a multi-channel counting experiment approach.A multilepton event consists of at least three isolated and prompt leptons (e, µ, τ h ), of which at least two must be electrons or muons ("light" leptons).A photon pair together with at least one lepton makes a diphoton+lepton event.Rare as the production of such multilepton and diphoton+lepton states in SM may be, the new phenomena targeted here are equally elusive.
In the A → Zh search, the multilepton and diphoton signal events can result from the WW, ZZ, ττ and γγ decays of the h, when accompanied by the appropriate decays of the W and Z bosons and the τ lepton.Five combinations of the Zh decays can result in a final state containing multileptons and one combination (Z → , h → γγ) leads to diphoton+lepton states with substantial rates.
For the t → ch search, three combinations in the decay chain tt → (bW)(ch) → (b ν)(ch) can lead to multilepton final states, namely h → WW * , h → ZZ * , and h → ττ.The bWch channel can also result in a diphoton+lepton final state when the Higgs boson decays to a photon pair.Finally, given the parent tt state, the amount of hadronic activity in the t → ch signal events is expected to be quite large.

Multilepton search channels
A three-lepton event must contain exactly three isolated and prompt leptons (e, µ, τ h ), of which two must be electrons or muons.Similarly, a four-lepton event must contain at least four leptons, of which three must be electrons or muons.With the goal of segregating SM backgrounds, these events are classified on the basis of the lepton flavor, their relative charges, as well as charge and flavor combinations and other kinematic quantities such as dilepton invariant mass and E miss T , as follows.
Events with τ h are grouped separately because narrow jets are frequently misidentified as τ h , leading to larger SM backgrounds for channels with τ h .Similarly, the presence of a b-tagged jet in an event calls for a separate grouping in order to isolate the tt background.
The next classification criterion is the maximum number of opposite-sign and same-flavor (OSSF) dilepton pairs that can be constructed in an event using each light lepton only once.For example, both µ + µ − µ − and µ + µ − e − are said to be OSSF1, and a µ + e − τ h would be OSSF0.Both e + e + µ − and µ + µ + τ h are OSSF0(SS), where SS additionally indicates the presence of samesigned electron or muon pairs.Similarly, µ + µ − e + e − is OSSF2.An event with an OSSF pair is said to be "on-Z" if the invariant mass of at least one of the OSSF pair is between 75 GeV and 105 GeV, otherwise it is "off-Z".An OSSF1 off-Z event is "below-Z" or "above-Z" depending on whether the mass of the pair is less than 75 GeV or more than 105 GeV, respectively.An on-Z OSSF2 event may be a "one on-Z" or a "two on-Z" event.
Finally, the three-lepton events are classified in five E miss T bins: < 50, 50-100, 100-150, 150-200, and >200 GeV and the four-lepton events are classified in four E miss T bins: < 30, 30-50, 50-100, and >100 GeV.This results in a total of 70 three-lepton channels and 72 four-lepton channels which are listed explicitly when we later present the tables of event yields and background predictions (Tables 1 and 2).

Diphoton + lepton search channels
A diphoton pair together with at least one lepton makes a diphoton+lepton event.The diphoton invariant mass of the h → γγ candidates must be between 120 and 130 GeV.The search channels are γγ , γγ τ h , γγ , and γγτ h .Depending on the relative dilepton flavor and invariant mass, the γγ events can be OSSF0, OSSF1 on-Z, or OSSF1 off-Z.The SM background decreases with increasing E miss T , therefore the events are further classified, when appropriate, in three bins: E miss T < 30, 30-50, and >50 GeV.
The t → ch signal populates the γγ and γγτ h channels but not the diphoton+dilepton channels.Since the t → ch signal events always contain a b quark from the conventional bW decay of one of the top quarks, the γγ and γγτ h search channels are further classified based on the presence of a b-tagged jet.For these channels, we also split the last E miss T bin into two: 50-100 and >100 GeV.
The overall diphoton+lepton channel count in this search is seven for γγ , three for γγ τ h , and eight each for γγ and γγτ h .They are listed explicitly when we present the tables of event yields and background predictions later (Tables 4 and 5).

Multilepton background estimation
Significant sources of multilepton SM background are Z + jets, diboson production (VV + jets; V = W, Z), tt production, and rare processes such as ttV + jets.The techniques we use here to estimate these backgrounds are identical to those used in Ref. [8] and are described briefly below.
WZ and ZZ diboson production can yield events with three or four intrinsically prompt and isolated leptons that can be accompanied by significant E miss T and H T .To estimate these background contributions we use a simulation validated after kinematic comparisons with appropriately enriched data samples.
Processes such as Z + jets and W + W − + jets can yield events with two prompt leptons.These can be accompanied by jets that may also contain leptons from the semileptonic decays of hadrons, or objects that are occasionally reconstructed as prompt leptons, leading to a threelepton SM background.Since the simulation of the rare fluctuations that lead to such a misidentified prompt lepton can be unreliable, we use the data with two reconstructed leptons to estimate this SM background using the number of isolated prompt tracks in the dilepton dataset.
The tt decay can result in two prompt leptons and is a source of background when the decay of one of the daughter b quarks reconstructs as the third prompt lepton candidate.This background is estimated from a tt Monte Carlo sample and using the probability of occurrence of a misidentified third lepton derived from data.
For search channels that contain τ h , we estimate the probability of a (sparse) jet misidentified as a τ h candidate by extrapolating the isolation distribution of the τ h candidates.Since the shape of this distribution is sensitive to the extent of jet activity, the extrapolation is carried out as a function of the hadronic activity in the sample as determined by the summed p T of all tracks as well as the leading jet p T in the event.
Finally, minor backgrounds from rare processes such as ttV + jets or SM Higgs production including its associated production with W, Z, or tt are estimated using simulation.

Diphoton+lepton background estimation
We use a 120-130 GeV diphoton invariant mass window to capture the h → γγ signal.With the requirement of at least one lepton in these diphoton+lepton channels, the SM background tends to be small and is estimated by interpolating the diphoton mass sidebands of the signal window.We assume the background distribution shape to be a decaying exponential as a function of the diphoton invariant mass over the 100-200 GeV mass range.
Figure 1 (Left) shows the exponential fit to the 100-120 and 130-200 GeV sidebands in the mass distribution for γγτ h events with E miss T < 30 GeV.We choose this sample to determine the decay coefficient because it is a high-statistics sample.This decay exponent is used for background determination in all diphoton channels, allowing only the normalization to float from channel to channel. Figure 1 (Right) shows an example of such a fit for the γγ sample with a 30-50 GeV E miss T requirement along with an exponential fit where both the decay exponent and normalization are allowed to float.We assign a 50% systematic uncertainty for background determination in the 120-130 GeV Higgs boson mass region.The figure also shows an amplified signal expected for m H = 300 GeV, assuming that the production cross section σ for m H = 300 GeV is equal to the Standard Model Higgs gluon fusion value of 3.59 pb at this mass given by the LHC Higgs Cross Section Working Group in Ref. [27], and a branching fraction B(H → hh) = 1.

Observations
Tables 1 and 2 list the observed number of events for the three-lepton and four-lepton search channels, respectively.The number of expected events from SM processes are also shown together with the combined statistical and systematic uncertainties.Table 3 lists the sources of systematic effects and the resultant uncertainties in estimating the expected events from the SM.All search channels share systematic uncertainties for luminosity, renormalization scale, PDF, and trigger efficiency.
The observations listed in the tables generally agree with the expectations within the uncertainties.Given the large number of channels being investigated simultaneously, certain deviations between observations and expected values are to be anticipated.We discuss one such deviation later in the context of the H search.
Figure 2 shows observations and background decomposition for some of the most sensitive Table 1: Observed (Obs.)yields and SM expectations (Exp.) for three-lepton events.See text for the description of event classification by the number and invariant mass of opposite-sign, same-flavor lepton pairs that are on-or below-Z (see Section 4.2), presence of τ h , tagged b jets, and the E miss T in the event.The 70 channels are exclusive.Table 2: Observed (Obs.)yields and SM expectation (Exp.) for four-lepton events.See text for the description of event classification by the number and invariant mass of opposite-sign, same-flavor lepton pairs that are on-or off-Z, presence of τ h , tagged b jets, and the total E miss T in the event.The 72 channels are exclusive.channels for the H → hh search.The amount of signal for m H = 300 GeV, as described above in the context of Fig. 1, is also shown.Figure 3 shows the same for the A → Zh search for m A = 300 GeV, assuming the same cross section and B(A → Zh) = 1.
(GeV)   The diphoton+lepton results are summarized in Tables 4 and 5.The observations agree with the expectations within the uncertainties.Table 4: Observed yields and SM expectations for diphoton+dilepton events.The diphoton invariant mass is required to be in the 120-130 GeV window.The ten channels are exclusive.7 Interpretation of results

Statistical procedure
No significant disagreement is found between observations and SM expectations.We derive limits on the production cross-section times branching fraction for the new physics scenarios under consideration, and use them to constrain parameters of the models.We set 95% CL upper limits on the cross sections using the modified frequentist construction CL [28,29].We compute the single-channel CL limits for each channel and then obtain the combined upper limit.

H → hh and A → Zh model-independent interpretations
Figure 4 (left) shows 95% CL observed and expected σB upper limit for the gluon fusion production of heavy scalar H, with the decay H → hh along with one and two standard deviation bands around the expected limits using only the multilepton channels.Figure 4 (right) shows the same using both multilepton and diphoton channels.In placing these model-independent limits, we explicitly assume that h is the recently discovered SM-like Higgs boson [1][2][3] particularly in regards to the branching fraction of its various decay modes as predicted in the SM.

Interpretation of results
For low masses, there is an almost two standard deviation discrepancy between the expected and observed 95% CL limits in Fig. 4. Its origin traces back to three four-lepton channels listed in Table 2, which can also be located in Fig. 2 (right).They consist of events with a τ h and three light leptons containing an off-Z OSSF dilepton pair, but not a b-tagged jet.The H → hh signal resides almost entirely in the 0-100 GeV range in E miss T which is spanned by these three channels collectively.The observed (expected) number of events is 11 (5.7 ± 1.7), 4 (2.4 ± 0.5) and 5 (2.6 ± 0.6) for E miss T in ranges 0-30, 30-50, and 50-100 GeV, respectively.Summing over the three channels, the observed count is 20 with an expectation of 10.7 ± 1.9, giving the probability of observing 20 events over the 0-100 GeV E miss T range to be approximately 2.2%.Systematic uncertainties and their correlations are taken into account when evaluating this probability.The observed discrepancy in the limits shown in Fig. 4   Next we probe the sensitivity to gluon fusion production of the heavy pseudoscalar A with the decay A → Zh. Figure 5 (left) shows 95% CL upper limits on σB for A → Zh search along with one and two standard deviation bands around the expected contour using only the multilepton channels.Figure 5 (right) shows the same signal probed with both multilepton and diphoton channels.The observed and expected exclusions are consistent.

Interpretations in the context of 2HDM
General models with two Higgs doublets may exhibit new tree-level contributions to flavorchanging neutral currents that are strongly constrained by low-energy experiments.Prohibitive flavor violation is avoided in 2HDM if all fermions of a given representation receive their masses through renormalizable Yukawa couplings to a single Higgs doublet, as in the case of supersymmetry.There are four such possible distinct assignments of fermion couplings in models with two Higgs doublets, the most common of which are called Type I and Type II models.In Type I models all fermions receive their masses through Yukawa couplings to a single Higgs doublet, while in Type II models the up-type quarks receive their masses through couplings to one doublet and down-type quarks and leptons couple to the second doublet.In either type, after electroweak symmetry breaking the physical Higgs scalars are linear combinations of these two electroweak Higgs doublets, so that fermion couplings to the physical  states depend on the type of 2HDM, the mixing angle α, and the ratio of vacuum expectation values tan β.We next present search interpretations in the context of Type I and Type II 2HDMs [5].In these models, the production cross sections for H and A as well as the branching fractions for them to decay to two SM-like Higgs bosons depend on parameters α and tan β.The mixing angle between H and h is given by α, whereas tan β gives the relative contribution of each Higgs doublet to electroweak symmetry breaking.In obtaining these model-dependent limits, the daughter h is assumed to be the recently discovered SM-like Higgs boson, but the branching fractions to its various decay modes are assumed to be dictated by the parameters α and tan β of the 2HDM, as described below.
We use the SUSHI [30] program to obtain the 2HDM cross sections.The branching fraction for SM-like Higgs boson are calculated using the 2HDMC [31] program.The 2HDMC results are consistent with those provided by the LHC Higgs cross section working group [32].A detailed list of couplings of H and A to SM fermions and massive gauge bosons in Type I and II 2HDMs has been tabulated in Ref. [6]. Figure 6 (top left and bottom left) shows observed and expected 95% CL upper limits for gluon fusion production of a heavy Higgs boson H of mass 300 GeV for Type I and Type II 2HDMs, respectively, along with the σB theoretical predictions (right) adopted from Ref. [33] for the two models.Figure 7 (top left and bottom left) shows similar results for the pseudoscalar A Higgs boson of mass 300 GeV.The branching fractions for the SM-like Higgs boson daughters of the H and A vary across the tan β versus cos(β − α) plane and are incorporated in the upper limit calculations.
Finally, we further improve constraints on the 2HDM parameters using the simultaneous null findings for the H and A. Figure 8 shows exclusion in tan β versus cos(β − α) plane for the combined gluon fusion signal for Type I (left) and Type II (right) 2HDMs, assuming H and A to be mass degenerate with a mass of 300 GeV.Once again, the branching fractions of the SM-like h daughters are allowed to vary across the plane.

t → ch search results
The t → ch signal predominantly populates diphoton+lepton channels with a b-tag and (no τ h ) multilepton channels that lack an OSSF dilepton pair or have an off-Z OSSF pair.Beyond the fact that the presence of charm quark increases the likelihood of an event being classified as being b-tagged, no special effort is made to identify the charm quark present in the signal.The observations and SM expectations for the ten most sensitive channels are listed in Table 6  along with the signal yield for a nominal value of B(t → ch) = 1%.No significant excess is observed.
The statistical procedure yield an observed limit of B(t → ch) = 0.56% and an expected limit of B(t → ch) = (0.65 +0.29  −0.19 )% from SM tt production followed by either t → ch or its chargeconjugate decay.The t → ch branching fraction is related to the left-and right-handed top flavor changing Yukawa couplings λ h tc and λ h ct , respectively, by B(t → ch) 0.29 ], so that the observed limit corresponds to a limit on the couplings of |λ h tc | 2 + |λ h ct | 2 < 0.14.
To facilitate interpretations in broader contexts [34], we also provide limits on B(t → ch) from individual Higgs boson decay modes.For this purpose, we assume the SM branching fraction for the Higgs boson decay mode [35] under consideration, and ignore other decay modes.Table 7 shows the results, illustrating the analysis sensitivity for the t → ch decay in each of the Higgs boson decay modes.

Summary
We have performed a search for the H → hh and A → Zh decays of heavy scalar (H) and pseudoscalar (A) Higgs bosons, respectively, which occur in the extended Higgs sector described by    the 2HDM.This is the first search for these decays carried out at the LHC.We used multilepton and diphoton final states from a dataset corresponding to an integrated luminosity of 19.5 fb −1 of data recorded in 2012 from pp collisions at a center-of-mass energy of 8 TeV.We find no Table 6: The ten most sensitive search channels for t → ch, along with the number of observed (Obs.), expected SM background (Exp.), and expected signal (Sig.)events (assuming B(t → ch) = 1%).The three-lepton channels are taken from Ref. [8], have H T < 200 GeV and do not contain a τ h .The stated uncertainties contain both systematic and statistical components.=0.23% 0.69% 0.81% (0.60-1.17)%Combined multileptons + diphotons 0.56% 0.65% (0.46-0.94)% significant deviation from the SM expectations and place 95% CL cross section upper limits of approximately 7 pb on σB for H → hh and 2 pb for A → Zh.We further interpret these limits in the context of Type I and Type II 2HDMs, presenting exclusion contours in the tan β versus cos(β − α) plane.

Channel
Using diphoton and multilepton search channels that are sensitive to the decay t → ch, we place an upper limit of 0.56% on B(t → ch), where the expected limit is 0.65%.This is a significant improvement over the earlier limit of 1.3% from the multilepton search alone [8].

Figure 1 :
Figure 1: Left: diphoton invariant mass distribution for γγτ h events with E miss T < 30 GeV with an exponential fit derived from the 100-120 and 130-200 GeV sidebands regions.Right: the same distribution for the γγ events with E miss T in 30-50 GeV range with an exponential fit (blue curve) where the exponent is fixed to the value obtained from the fit shown in the leftfigure.Also shown for comparison purposes is a direct fit (magenta curve) to the shown data distribution.An example signal distribution (in red) shows that the signal is well-contained in the 120-130 GeV window.

Figure 2 :
Figure2: The E miss T distributions for four-lepton events with an off-Z OSSF1 dilepton pair, no btagged jet, and no τ h (left), and one τ h (right).These non-resonant (off-Z) channels are sensitive to the H → hh signal which is shown stacked on top of the background distributions.

Figure 3 :
Figure3: The E miss T distributions for four-lepton events without b-tagged jets which contain an on-Z OSSF1 dilepton pair and one τ h (left), and an OSSF2 dilepton pairs with one Z candidate and no τ h (right).These resonant (containing a Z) channels are sensitive to the A → Zh signal which is shown stacked on top of the background distributions.
is thus consistent with a broad fluctuation in the observed E miss T distribution.Given the large number of channels under scrutiny in this search, fluctuations at this level are to be expected.No such deviation is observed in the E miss T distribution for other search channels.

Figure 4 :
Figure 4: Left: observed and expected 95% CL σB limits for gluon fusion production of H and the decay H → hh with one and two standard deviation bands shown.These limits are based only on multilepton channels.The h branching fractions are assumed to have SM values.Right: the same, but also including diphoton+lepton channels.

Figure 5 :
Figure 5: Left: observed and expected 95% CL σB limits for gluon fusion production of A and the decay A → Zh with one and two standard deviation bands shown.These limits are based only on multilepton channels.The h branching fraction are assumed to have SM values.Right: the same, but also including diphoton+lepton channels.

Figure 6 :
Figure 6: Left: observed and expected 95% CL upper limits for gluon fusion production of a heavy Higgs boson H of mass 300 GeV as a function of parameters tan β and cos(β − α) of the Type I (upper) and II (lower) 2HDM.The parameters determine the H production cross section as well as the branching fractions B(H → hh) and B(h → WW * , ZZ * , ττ, γγ), which are relevant to this search.Right: the σB(H → hh) contours for Type I (upper) and II (lower) 2HDM adopted from Ref.[33].The excluded regions are either below the open limit contours or within the closed ones.

Figure 7 :
Figure 7: Left: observed and expected 95% CL upper limits for gluon fusion production of an A boson of mass 300 GeV as a function of parameters tan β and cos(β − α) of the Type I (upper)and II (lower) 2HDM.The parameters determine the A production cross section as well as the branching fractions B(A → Zh) and B(h → WW * , ZZ * , ττ, γγ) which are relevant to this search.Right: the σB(A → Zh) contours for Type I (upper) and II (lower) 2HDM adopted from Ref.[33].The excluded regions are below the open limit contours.

Figure 8 :
Figure 8: Combined observed and expected 95% upper limits for gluon fusion production of a heavy Higgs boson H and A of mass 300 GeV for Type I (left) and Type II (right) 2HDM as a function of parameters tan β and cos(β − α).The parameters determine the H and A production cross sections as well as the branching fractions B(H → hh), B(A → Zh), and B(h → WW * , ZZ * , ττ, γγ), which are relevant to this search.

Table 3 :
A compilation of significant sources of systematic uncertainties in the event yield estimation.Note that a given uncertainty may pertain only to specific sources of background.The listed values are representative and the impact of an uncertainty varies from search channel to channel.

Table 5 :
Observed yields and SM expectations for diphoton+single lepton events.The diphoton invariant mass is required to be in the 120-130 GeV window.The eight channels are exclusive.