Search for leptoquarks coupled to third-generation quarks in proton-proton collisions at $\sqrt{s} =$ 13 TeV

Three of the most significant measured deviations from standard model predictions, the enhanced decay rate for B $\to$ D$^{(*)}\tau\nu$, hints of lepton universality violation in B $\to$ K$^{(*)}\ell\ell$ decays, and the anomalous magnetic moment of the muon, can be explained by the existence of leptoquarks (LQs) with large couplings to third-generation quarks and masses at the TeV scale. The existence of these states can be probed at the LHC in high energy proton-proton collisions. A novel search is presented for pair production of LQs coupled to a top quark and a muon using data at a center-of-mass energy of 13 TeV, corresponding to an integrated luminosity of 35.9 fb$^{-1}$, recorded by the CMS experiment. No deviation from the standard model prediction has been observed and scalar LQs decaying exclusively into t$\mu$ are excluded up to masses of 1420 GeV. The results of this search are combined with those from previous searches for LQ decays into t$\tau$ and b$\nu$, which excluded scalar LQs below masses of 900 and 1080 GeV. Vector LQs are excluded up to masses of 1190 GeV for all possible combinations of branching fractions to t$\mu$, t$\tau$ and b$\nu$. With this analysis, all relevant couplings of LQs with an electric charge of -1/3 to third-generation quarks are probed for the first time.


1
The standard model of particle physics has been outstandingly successful in describing most fundamental physical phenomena. However, significant deviations from the predictions of the standard model (SM) have been observed in measurements of rare decays of B mesons. In particular, deviations have been seen in the values of the ratio R D ( * ) , defined as the ratio of the B → D ( * ) τν branching fraction to the B → D ( * ) µν branching fraction. These deviations from the SM were first reported by the BaBar [1,2] and Belle [3][4][5] Collaborations and have been confirmed by the LHCb Collaboration [6, 7] with a combined significance of about four standard deviations [8]. The ratios of the branching fractions of B → K ( * ) µµ to B → K ( * ) ee, R K and R K * , as measured by the LHCb Collaboration [9-12], show departures from lepton universality by 2.6 and 2.4 standard deviations, respectively. The measurement of the muon anomalous magnetic moment a µ , one of the most precisely measured quantities in particle physics [13], also deviates from the SM prediction by 3.5 standard deviations [14]. These anomalies are among the most significant deviations from the SM observed so far.
The existence of leptoquarks (LQs) with masses at the TeV scale and large couplings to thirdgeneration quarks [15][16][17][18][19][20][21][22][23][24][25] has been proposed as a possible explanation for one, two, or all of these deviations. Leptoquarks are hypothetical particles that can decay to SM quarks and leptons. They are triplets with respect to the strong interaction, have fractional electric charge, and can be either scalar (spin 0) or vector (spin 1) particles. Many extensions to the SM, among them grand unification [26][27][28], technicolor [29,30], and compositeness models [31,32], predict the existence of these particles. The effective Buchmüller-Rückl-Wyler model [33] incorporates the assumption that LQ interactions with SM fermions are renormalizable and gauge invariant, leading to restrictions on the allowed quantum numbers of LQs [34].
At the CERN LHC, pair production of LQs is possible via gluon-gluon fusion or quark-antiquark annihilation, allowing direct searches to be performed. Single LQ production via quark-gluon scattering is subdominant for LQs coupled to heavy quarks, as it requires a heavy quark in the initial state. The pair production cross section depends on the mass of the scalar LQ and is known at next-to-leading order (NLO) precision [48]. The pair production cross section for vector LQs has been calculated at leading order (LO) [49] and is much larger than the scalar LQ cross section. The cross section for vector LQs depends on an additional parameter κ, which is a dimensionless coupling and takes a value of κ = 1 in the Yang-Mills case and κ = 0 in the minimal coupling case.
The central feature of the CMS apparatus is a superconducting solenoid of 6 m internal diameter, providing a magnetic field of 3.8 T. Within the solenoid volume are a silicon pixel and strip tracker, a lead tungstate crystal electromagnetic calorimeter, and a brass and scintillator hadron calorimeter, each composed of a barrel and two endcap sections. Forward calorimeters extend the pseudorapidity (η) coverage provided by the barrel and endcap detectors. Muons are detected in gas-ionization chambers embedded in the steel flux-return yoke outside the solenoid. A more detailed description of the CMS detector, together with a definition of the coordinate system used and the relevant kinematic variables, can be found in Ref. [50].
This analysis uses data recorded by the CMS detector in pp collisions at a center-of-mass energy of 13 TeV in 2016. Online, potential signal events are required to pass a single-muon trigger that selects isolated muon candidates with transverse momentum p T > 24 GeV [51]. Data recorded by single electron triggers are used in background-enriched control regions (CRs). The data correspond to an integrated luminosity of 35.9 fb −1 .
Signal events of pair-produced LQs with prompt decays to tµ are simulated with the PYTHIA 8.205 [52,53] Monte Carlo program at LO for mass values ranging from 200 to 2000 GeV. The POWHEG [54][55][56][57][58][59][60][61] v1 generator is used to simulate background events resulting from the production of single top quarks in the tW-channel at NLO. The POWHEG v2 generator is used for single top production in the t-channel and for simulating tt production at NLO. Single top quark production in the s-channel, tt production in association with a heavy gauge boson (tt + V), and the production of a W boson with additional jet radiation are simulated with MADGRAPH5 aMC@NLO (v2.2.2) [62] at NLO. Events from Drell-Yan (DY) production with additional jet radiation are simulated with MADGRAPH5 aMC@NLO at LO and an NLO K factor is applied to the LO DY+jets production cross section. The simulation of the production of two heavy gauge bosons with additional jet radiation is performed at NLO with MAD-GRAPH5 aMC@NLO and POWHEG v2. Events in which jets are produced through the strong interaction only, referred to as quantum chromodynamic multijet events, are simulated with PYTHIA at LO.
Parton showers in the simulated W boson production events and DY events with additional jet radiation are matched to the matrix element calculation with the FxFx [63] and MLM [64] algorithms, respectively. The parton shower and hadronization process is simulated with PYTHIA. The NNPDF3.0 [65] parton distribution functions (PDFs) at LO and NLO are used for processes simulated at LO and NLO, respectively. The underlying event tune CUETP8M2T4 [66] is used for the simulation of tt and single top quark production via the t-channel, all other processes are generated using CUETP8M1 [67,68]. All simulated event samples include the simulation of additional inelastic pp interactions within the same or adjacent bunch crossings (pileup). The detector response is simulated with the GEANT4 package [69,70]. Simulated events are processed through the software chain used for collision data and are reweighted to match the observed distribution of the number of pileup interactions in data.
The CMS experiment uses a particle-flow (PF) event reconstruction algorithm [71], which makes use of an optimized combination of information from the various elements of the CMS detector. The reconstructed vertex with the largest value of summed physics object p 2 T is taken to be the primary pp interaction vertex. The physics objects here are the objects returned by a jet finding algorithm [72,73] applied to all charged tracks associated with the vertex, plus the associated missing transverse momentum, taken as the negative vector p T sum of those jets. More details are given in Ref. [74]. All detected particles are reconstructed either as electrons, muons, photons, charged hadrons, or neutral hadrons. In this analysis, electrons and muons are required to have p T ≥ 30 GeV, |η| ≤ 2.4, and to be isolated. The isolation [75, 76] is defined as the summed p T of all neutral particles and charged hadrons in a cone with radius ∆R = √ (∆η) 2 + (∆φ) 2 , φ being the azimuthal angle in radians, of 0.4 (for muons) or 0.3 (for electrons) around the lepton. The sum is corrected for the contribution of neutral pileup inside the cone. Jets are clustered from charged and neutral PF candidates using the anti-k T jet-clustering algorithm [72,73] with a distance parameter of 0.4. Charged PF candidates originating from vertices other than the primary vertex are not clustered. A jet energy correction (JEC) is applied [77] to account for remaining contributions arising from a different vertex than the primary one as well as for non-uniformity of the jet response in η and non-linearity in p T . Finally, a correction is applied to account for the residual differences in the jet response between data and simulated events. The jet energy resolution (JER) in simulated events is smeared to match the wider resolution in data. All jets are required to have p T ≥ 30 GeV and |η| ≤ 2.4. The combined secondary vertex v2 [78] algorithm is used to identify jets originating from bottom quarks (b-tagged jets). The loose working point is chosen, which has an efficiency of about 90% and a mistag rate of approximately 10%. The missing transverse momentum, p miss T , is calculated as the magnitude of the negative vectorial p T sum of all PF candidates in an event. Both the jet energy scale and resolution corrections are propagated to the calculation of p miss T . Offline, events are required to contain at least two muons and at least two jets, of which at least one must be b-tagged. By requiring the invariant mass of each pair of muons in an event to exceed the Z boson mass by at least 20 GeV, events arising from the production of a Z boson with additional jet radiation are suppressed. As the decay of heavy LQs is expected to produce highly energetic leptons and jets, minimum values of S lep T and S T of 200 and 350 GeV are required, respectively. Here, S lep T is the scalar p T sum of all selected muons and electrons and S T is defined as the scalar sum of S lep T , p miss T , and the p T of all selected jets. The phase space region resulting from these selection criteria is referred to as the signal region (SR) in the following.
Two categories of events are defined, based on the number of muons or electrons. If at least three such charged leptons are present, of which at least two are muons, and at least one pair of muons with opposite electric charge is found, the event falls into category A. Category B contains all remaining events in the SR. In category A, the LQ mass M rec LQ is reconstructed under the assumption that one of the top quarks decays into the leptonic final state with a muon or an electron (leptonic top) and the other one decays into the hadronic final state (hadronic top). The distribution of M rec LQ is used for the final statistical analysis in this category, while the distribution of S T is used for this purpose in category B.
For each event, the leptonic top quark candidates are constructed from permutations of one or more of the seven p T -leading jets, one of the three p T -leading muons or the p T -leading electron, and p miss T . The hadronic top quark candidates are constructed using all permutations of jets not assigned to the leptonic top quark. The LQ candidates are assembled from top quark candidates and the two p T -leading muons that have not been associated to the leptonic top quark. The muon charge is used when assigning it to one of the top quark candidates. In events with more than two muons, all possible permutations of muons are considered. A χ 2 variable that takes into account the invariant mass of each top quark candidate and the relative mass difference between the two LQ candidates is then used to select the best pair of LQ candidates for each event. In order to provide a more accurate SM background prediction in category A, which contains a minimum of three charged leptons, the misidentification rate of electrons and muons is measured using jets in a DY+jets enriched CR in data. The resulting data-to-simulation correction factors are applied to simulation for each misidentified charged lepton in a given event in the SR.
The contributions from the dominant SM backgrounds in category B, the production of tt and DY+jets events, are estimated simultaneously in a data-driven procedure. A CR similar to the SR is defined by requiring a minimum of two electrons without additional muons. The invariant mass of any pair of electrons must be at least 20 GeV above the Z boson mass and all other SR requirements have to be fulfilled for the selected events. We correct for small differences in the distribution of S T between the SR and the CR with an extrapolation function α(S T ), which is derived from simulated tt and DY+jets events by fitting both S T distributions with an empirical functional form to obtain smoothed distributions that are then used to compute the ratio. The number of data events in the CR, after all simulated minor backgrounds have been subtracted, is multiplied by α(S T ) to extrapolate into the SR. Using the ratio of the fitted functions results in a significantly smaller impact of systematic uncertainties on the estimated backgrounds. The THETA software package [91] is used to perform a maximum-likelihood template fit to the binned M rec LQ and S T distributions for the background and to extract the cross section of a potential signal. The statistical uncertainties in the SM backgrounds and the signal, as well as all systematic uncertainties, are taken into account as nuisance parameters in the fit. The uncertainty in the luminosity is assigned a log-normal prior distribution, for all other systematic uncertainties a Gaussian prior is used. The statistical uncertainty in the predicted background and the signal is taken into account by defining one additional nuisance parameter with a Gaussian distribution for each bin. A flat prior distribution is assumed for the signal cross section. The data are found to be compatible with the SM prediction in both categories. The distributions of M rec LQ and S T after the background-only fit are shown in Fig. 1. A Bayesian method [91-93] is used to set upper limits at 95% confidence level (CL) on the cross section for pair production of LQs decaying into a top quark and a muon. Pseudo-experiments are performed to determine the median along with the regions expected to contain 68 and 95% of the distribution of limits under the background-only hypothesis.