Search for evidence of the type-III seesaw mechanism in multilepton final states in proton-proton collisions at sqrt(s) = 13 TeV

A search for a signal consistent with the type-III seesaw mechanism in events with three or more electrons or muons is presented. The data sample consists of proton-proton collisions at sqrt(s) = 13 TeV collected by the CMS experiment at the LHC in 2016 and corresponds to an integrated luminosity of 35.9 inverse femtobarns. Selection criteria based on the number of leptons and the invariant mass of opposite-sign lepton pairs are used to distinguish the signal from the standard model background. The observations are consistent with the expectations from standard model processes. The results are used to place limits on the production of heavy fermions of the type-III seesaw model as a function of the branching ratio to each lepton flavor. In the scenario of equal branching fractions to each lepton flavor, heavy fermions with masses below 840 GeV are excluded. This is the most sensitive probe to date of the type-III seesaw mechanism.


1
The discovery of neutrino oscillations established that neutrinos have mass [1]. Among proposed extensions of the Standard Model (SM) that explain neutrino mass, models based on the seesaw mechanism are an appealing possibility [2][3][4][5][6][7][8][9][10]. The seesaw mechanism introduces new heavy particles that couple to leptons and to a Higgs boson (H) doublet, and accounts for the smallness of neutrino masses [11][12][13]. Within the type-III seesaw model [8], the neutrino is considered a Majorana particle whose mass arises from the mediation of new massive fermions that form an SU(2) triplet of heavy Dirac charged leptons Σ ± and a heavy Majorana neutral lepton Σ 0 . In proton-proton (pp) collisions, these massive fermions may be pair-produced through leading order electroweak interactions, in both charged-charged and charged-neutral pairs. The renewed interest in the type-III seesaw models [14][15][16][17][18][19] emphasizes the importance of exploring such signatures at the CERN LHC.
In the model used in this analysis [22], the Σ ± and Σ 0 decays are prompt, and gauge invariance ensures that they are degenerate in mass at tree level. The branching fraction (B ) of a heavy fermion to a lepton of flavor is proportional to |V | 2 /(|V e | 2 + |V µ | 2 + |V τ | 2 ), where V is the heavy-light fermion mixing angle. A prior CMS search using 7 TeV data excluded heavy fermions with mass (m Σ ) below 179 GeV [23], probing a scenario where the branching fractions to all lepton flavors are equal (flavor-democratic scenario, B e = B µ = B τ ). Using 8 TeV data, the ATLAS Collaboration similarly excluded m Σ below 335 GeV, assuming that the Σs decay only to the first two generations of leptons [24]. In this Letter, we probe all values of B for each flavor of , subject to the constraint B e + B µ + B τ = 1.
We pursue a search in final states with at least three charged leptons. The backgrounds can be classified as the irreducible ones due to the production and decay of dibosons (WZ and ZZ production), and the reducible ones due to the leptonic decay of Z+jets or tt+jets accompanied by leptons originating from heavy quark decay, or from misidentification of jets as leptons. In addition, small irreducible contributions to the background arise from other SM processes such as ttW, ttZ, triboson (ZZZ, WWZ etc.), and Higgs boson production.
The central feature of the CMS apparatus is a superconducting solenoid of 6 m internal diameter, providing a magnetic field of 3.8 T. Within the solenoid volume are a silicon pixel and strip tracker, a lead tungstate crystal electromagnetic calorimeter (ECAL), and a brass and scintillator hadron calorimeter, each composed of a barrel and two endcap sections. Forward calorimeters extend the pseudorapidity (η) coverage provided by the barrel and endcap detectors. Muons are detected in gas-ionization chambers embedded in the steel flux-return yoke outside the solenoid. A detailed description of the CMS detector can be found in Ref. [25].
The data sample consists of pp collisions at √ s = 13 TeV collected by the CMS experiment at the LHC in 2016, corresponding to an integrated luminosity of 35.9 fb −1 . The data for this search are collected using several dilepton triggers based on a two-level trigger system [26]. The double-electron (double-muon) trigger requires two loosely isolated electrons (muons). The transverse momentum (p T ) thresholds on the leading (subleading) lepton are 23 (12) GeV for the double-electron, and 17 (8) GeV for the double-muon trigger, respectively. We also use two muon/electron cross-trigger combinations that require a p T > 23 GeV loosely isolated muon (electron) and a p T > 8 GeV loosely isolated electron (muon).
The signal is simulated using MADGRAPH5 aMC@NLO v2.2.2 [27] at leading order precision using the NNPDF30 lo as 0130 nf 4 [28] parton distribution functions (PDFs). The signal cross section is calculated at next-to-leading order (NLO) plus next-to-leading logarithmic accuracy, assuming that the heavy leptons, Σ ±,0 , are SU (2)  The response of the CMS detector is simulated using dedicated software based on the GEANT4 toolkit [40]. The presence of multiple pp interactions in the same or adjacent bunch crossing (pileup) [41] is incorporated by simulating additional interactions that are both in-time and out-of-time with the collision.
Events satisfying the trigger are required to pass additional offline selections. Electrons and muons must satisfy p T > 10 GeV and |η| < 2.5 (|η| < 2.4 for muons). Electrons are reconstructed using Gaussian Sum Filter tracks matched to ECAL superclusters [42]. Muons are reconstructed using the particle-flow (PF) algorithm [43, 44] that utilizes measurements from the tracker, calorimeter and muon systems to reconstruct candidate muons, photons, and hadrons. Tracks of candidate electrons and muons must satisfy quality requirements and should match geometrically with either energy deposits in the ECAL (for electrons) or tracks in the muon detectors (for muons).
Jets are reconstructed from the PF candidates and electrons using the anti-k T algorithm [45] with distance parameter R = 0.4. Jets are required to have p T ≥ 30 GeV and |η| ≤ 3.0 and satisfy quality criteria that remove contributions from pileup. The missing transverse momentum vector p miss T is defined as the projection onto the plane perpendicular to the beam axis of the negative vector sum of the momenta of all reconstructed PF objects in an event [46]. Its magnitude is referred to as p miss T . Jet energy corrections are applied to all jets and also propagated to the calculation of p miss T . The reducible backgrounds include leptons inside or near jets and leptons from heavy quark decays. In addition hadrons that reach the muon detectors, or hadronic showers with large electromagnetic energy fractions can be misidentified as leptons. Henceforth we refer to all leptons not arising from vector boson decay (either directly or via an intermediate tau lepton) as misidentified leptons. Backgrounds from misidentified leptons are significantly reduced by an isolation requirement comparing the p T of a lepton to the summed p T of particles in its neighborhood. We utilize relative isolation, which is defined as the scalar p T sum, normalized to the lepton p T , of photon and hadron PF candidates within a cone around the lepton. This relative isolation is required to be less than 25% within a cone of size R = √ (∆η) 2 + (∆φ) 2 = 0.4 for muons, where φ is azimuthal angle in radians. For electrons, the relative isolation within a cone of R = 0.3 must be less than 7% (8%) for the reconstructed energy deposits situated in the ECAL barrel [|η| < 1.479] (ECAL endcap [|η| > 1.479]). Charged tracks returned by a jet finding algorithm [45,47] and the corresponding p miss T are associated with each reconstructed vertex. The vertex with the largest value of Σp 2 T over the charged tracks and the p miss T is taken to be the primary pp interaction vertex. All isolation quantities are corrected for contributions from particles originating from pileup vertices. Electrons and muons must satisfy |d z | < 0.1 cm and |d xy | < 0.05 cm, where d z and d xy are the longitudinal and transverse impact parameters of leptons with respect to the primary vertex, respectively. For electrons in the ECAL endcap, we relax these requirements to |d z | < 0.2 cm and |d xy | < 0.1 cm. Table 1: The signal regions summarized in terms of the number of leptons, the presence of an OSSF pair, and the kinematic variable used for signal discrimination. Each region is divided into eight bins in the kinematic variable, leading to 48 independent signal regions. Additional criteria based on p miss T ensure that signal and control regions do not overlap.
For the two leading leptons we apply offline p T thresholds of 25 and 15 GeV. Additional leptons must satisfy p T > 10 GeV. With these thresholds, the trigger efficiency for trilepton events is close to 100%. Events containing an opposite-sign same-flavor (OSSF) lepton pair with mass below 12 GeV are vetoed to reduce background from low-mass resonances. We reject trilepton events containing an OSSF pair with mass below 81 GeV when the trilepton mass is within a Z boson mass window (91 ± 10 GeV). This suppresses events from the Z → * → γ background process, where the photon converts into two additional leptons, one of which is lost.
We classify the selected multilepton events into statistically independent search channels using the number of leptons and OSSF pairs, and other kinematic quantities as described below. The number of OSSF pairs is counted using each lepton once. For example, µ + µ + e − is tagged as OSSF0, while µ + µ − e + e − is tagged as OSSF2. We further classify events as "on-Z", "below-Z", and "above-Z", based on the presence and mass of at least one OSSF pair relative to a Z boson mass window, defined as the range 81-101 GeV. In cases of ambiguity (such as µ + µ − µ − ), the OSSF pair with its mass closest to that of the Z boson takes precedence. We define L T as the scalar sum of all charged lepton p T s, and we use L T + p miss T as the main discriminating variable. This choice allows us to maintain sensitivity to decay modes such as Σ ± → ± Z → ± ± ∓ with large charged-lepton momenta, as well as Σ 0 → Hν → WWν, where charged lepton p T s are lower but the neutrinos contribute to p miss T . For events with OSSF1 and on-Z, the sensitivity is improved by considering the transverse mass, M T = (2p miss T p T [1 − cos( p miss T , p T )]) 1/2 using the lepton that is not part of the OSSF pair. Events are grouped in eight bins of width 150 GeV and 100 GeV for L T + p miss T and M T , respectively. In each case the highest bin includes overflow events.
Altogether, six event categories are considered with eight bins of L T + p miss T in (i) three leptons, OSSF1, above-Z; (ii) three leptons, OSSF1, below-Z; (iii) three leptons, OSSF0; (iv) four or more leptons, OSSF1; (v) four or more leptons, OSSF2; and eight bins of M T in (vi) three leptons, OSSF1, on-Z. The 48 mutually exclusive signal regions are summarized in Table 1.
The irreducible backgrounds are estimated using simulation samples. The dominant processes are WZ and ZZ production and are normalized to data using dedicated control regions. The WZ control sample consists of events with exactly three leptons, tagged as OSSF1 on-Z, satisfying 50 < p miss T < 100 GeV and M T > 30 GeV. The ratio of the predicted WZ rate to data (after correcting for backgrounds other than WZ) is 1.15 ± 0.08, where the uncertainty includes statistical and systematic contributions. The ZZ control sample consists of events with exactly four leptons, OSSF2, on-Z for both pairs, and with p miss T < 50 GeV. The ratio of the predicted ZZ rate to data is 1.25 ± 0.06. These factors are used to normalize the WZ and ZZ background estimates. We also use these control regions to verify the modeling of kinematic distributions such as L T + p miss T and M T . The "rare" irreducible backgrounds from triboson, ttW, ttZ, and Higgs boson processes are estimated from simulated samples using the theoretical cross sections.
The misidentified lepton background is estimated from observations using a 3-dimensional implementation of a matrix method [48] in which probabilities (rates) of prompt and misidentified leptons satisfying a loose lepton selection to also pass a tight lepton selection are measured in data using a dilepton selection for prompt leptons, and a trilepton signal-depleted selection (OSSF1, on-Z, p miss T < 50 GeV) for misidentified leptons. The rates are parametrized as a function of lepton p T , η and the number of PF candidates around the lepton. The rate measurements are dominated by Z+jets events, and are corrected using simulation to an average of Z+jets and tt+jets events.
A residual "conversion" background is also estimated using the data. When a final-state radiation photon from a lepton converts asymmetrically, often only one of the resultant leptons is reconstructed in the detector. To estimate this background, we count trilepton events where the trilepton mass is on-Z, but the dilepton mass is below-Z. These events are compared with dilepton+photon events where the dilepton invariant mass is below-Z, but the combined invariant mass of the two leptons and the photon is on-Z. Photons suitable for modeling conversions are selected by ignoring photons that are far away from a lepton or are close to a jet. The photon conversion rates are 3% to 5% depending upon the lepton type and p T . The conversion background in a signal region is estimated by scaling the observed number of dilepton+photon events with these rates.
As an example of the complete background estimation, Fig. 1 shows the background predictions and the observed events with three leptons, OSSF1, above-Z, and for events with four or more leptons, OSSF1, along with systematic uncertainties. The WZ (ZZ) background has a systematic uncertainty of 7% (5%) arising from the normalization factor measurement. The rare backgrounds have an uncertainty of 50% in the theoretical cross section and 2.5% due to the integrated luminosity measurement. The misidentified background has an uncertainty  Figure 2: The 95% confidence level upper limits on the cross section for production of heavy fermion pairs (Σ 0 Σ + , Σ 0 Σ − , and Σ + Σ − ). Also shown is the theoretical prediction for the cross section of the Σ pair production via the type III seesaw mechanism, with its uncertainty. In the flavor-democratic scenario (B e = B µ = B τ ), heavy fermion pair production is excluded for masses below 840 GeV. of 30%, which is derived by varying the measured rates up and down for each lepton flavor within their respective uncertainties. This uncertainty is dominated by the differences in rates in Z+jets and tt+jets events. The conversion background has an uncertainty of 30% arising from contamination by nonconversion backgrounds in the control sample used to measure the conversion rate.
Corrections are applied to the simulated background events to account for differences from data in lepton identification and isolation efficiencies, dilepton trigger efficiencies, as well as in the overall scale and resolution of energy and momentum measurements for electrons, muons, and jets. The minor uncertainties due to these corrections are considered separately and result in a 1-10% variation of the simulation-based background yields across all signal regions.
The uncertainties in the signal acceptance due to the PDF uncertainties are estimated by utilizing the replica weights of the NNPDF30 lo as 0130 nf 4 PDF set [49], and are found to be less than 3% in all signal bins. The signal yields also have uncertainties due to the integrated luminosity measurement (2.5%) and the the corrections applied to the simulation samples mentioned in the previous paragraph.
We observe no statistically significant excess in the various signal regions that we probe. We calculate a 95% confidence level (CL) upper limit on the sum of cross sections for the production of heavy fermion pairs (Σ 0 Σ + , Σ 0 Σ − , and Σ + Σ − ) using the CL s criterion [50][51][52][53]. We consider all sources of systematic and statistical uncertainties as nuisance parameters, with log-normal and gamma prior distributions, respectively, and perform a simultaneous fit across all search regions. Figure 2 shows the upper limits on the cross section in the flavor-democratic scenario where the observed (expected) limit on masses of heavy fermions is 840 (780) GeV. The signal cross section has an uncertainty of approximately 5-15% due to choices of PDFs, and factorization and renormalization scales in the mass range we consider. Figure 3 shows the expected and observed excluded mass limits in the B e -B τ plane (note that B µ = 1 − [B e + B τ ]). The limits are the most restrictive when the Σ branching fraction to either electron or muon flavor is maximal, and become less restrictive as the branching fraction to τ flavor increases.
In summary, a search has been performed for type-III seesaw heavy fermions in multilepton final states using 35.9 fb −1 of pp collision data at √ s = 13 TeV, collected using the CMS detector. No significant discrepancies are observed between the data and the standard model prediction. Assuming degenerate heavy fermion masses, previously unexplored regions of the signal model as a function of the branching ratio of heavy fermions to each lepton flavor are excluded. In the lepton-flavor democratic scenario (B e = B µ = B τ ), heavy fermions with mass below 840 GeV are excluded at 95% confidence level. In the τ-phobic case (B τ = 0, B e + B µ = 1) the mass limits range from 900 GeV in the pure electron scenario to 930 GeV in the pure muon scenario. In the e/µ-phobic case (B τ = 1, B e + B µ = 0) the mass limit is 390 GeV. These are the strongest constraints to date on the mass of heavy fermions associated with the type-III seesaw mechanism.

Acknowledgments
We congratulate our colleagues in the CERN accelerator departments for the excellent performance of the LHC and thank the technical and administrative staffs at CERN and at other CMS institutes for their contributions to the success of the CMS effort. In addition, we gratefully acknowledge the computing centers and personnel of the Worldwide LHC Computing Grid for delivering so effectively the computing infrastructure essential to our analyses. Finally, we acknowledge the enduring support for the construction and operation of the LHC and the CMS detector provided by the following funding agencies:     [41] CMS Collaboration, "Pileup removal algorithms", CMS Physics Analysis Summary CMS-PAS-JME-14-001, 2014.