Combined search for the quarks of a sequential fourth generation

Results are presented from a search for a fourth generation of quarks produced singly or in pairs in a data set corresponding to an integrated luminosity of $5{\mathrm{fb}}^{-1}$ recorded by the CMS experiment at the LHC in 2011. A novel strategy has been developed for a combined search for quarks of the up and down type in decay channels with at least one isolated muon or electron. Limits on the mass of the fourth-generation quarks and the relevant Cabibbo-Kobayashi-Maskawa matrix elements are derived in the context of a simple extension of the standard model with a sequential fourth generation of fermions. The existence of mass-degenerate fourth-generation quarks with masses below 685 GeV is excluded at 95% confidence level for minimal off-diagonal mixing between the third- and the fourth-generation quarks. With a mass difference of 25 GeV between the quark masses, the obtained limit on the masses of the fourth-generation quarks shifts by about $\pm 20\mathrm{GeV}$. These results significantly reduce the allowed parameter space for a fourth generation of fermions.

Figure 1

The $S_{\mathrm{T}}$ distribution for the subsamples with two $b$ jets and one $W$ boson (a), one $b$ jet and three $W$ bosons (b), two $b$ jets and two $W$ bosons (c), and the $m_{bW}$ distribution for the subsample with two $b$ jets and two $W$ bosons (d). The data distributions of these observables are compared to their expectation from the simulation assuming the fitted nuisance parameters. The fitted values of the nuisance parameters represent the systematic shifts that are applied on the simulation to fit the data in the background-only hypothesis. As an illustration, the total uncertainty band is shown around the simulated expected distribution before taking into account the fitted values of the nuisance parameters. The expected distribution for a signal is shown for two different values of the $V_{\mathrm{CKM}}^{4\times 4}$ parameter $A$ and for $b^{\prime }$ and $t^{\prime }$ masses of 550 GeV. The cross section of the signal in the plots is scaled by a factor of eight for visibility. The last bin in all the histograms includes the overflow. We do not expect much signal for $A=1$ in (a), because the subsample with two $b$ jets and one $W$ boson is mainly sensitive to single $t^{\prime }$-quark production.

Figure 2

Top: Exclusion limit on $m_{t^{\prime }}=m_{b^{\prime }}$ as a function of the $V_{\mathrm{CKM}}^{4\times 4}$ parameter $A$. The parameter values below the solid line are excluded at 95% CL. The inner (outer) band indicates the 68% (95%) confidence interval around the expected limit. The slope indicates the sensitivity of the analysis to the $t^{\prime }b$ and $t{b}^{\prime }$ processes. Bottom: For a $V_{\mathrm{CKM}}^{4\times 4}$ parameter value $A\sim 1$, the exclusion limit on $m_{t^{\prime }}$ versus $m_{t^{\prime }}-m_{b^{\prime }}$ is shown. The exclusion limit is calculated for mass differences up to 25 GeV. The existence of up-type fourth-generation quarks with mass values below the observed limit are excluded at the 95% CL.