Search for the highly suppressed decays $B^{-}\to K^{+}{\pi }^{-}{\pi }^{-}$ and $B^{-}\to K^{-}{K}^{-}{\pi }^{+}$

We report a search for the decays $B^{-}\to K^{+}{\pi }^{-}{\pi }^{-}$ and $B^{-}\to K^{-}{K}^{-}{\pi }^{+}$, which are highly suppressed in the standard model. Using a sample of $(467\pm 5)\times {10}^6$ $B\overline{B}$ pairs collected with the BABAR detector, we do not see any evidence of these decays and determine 90% confidence level upper limits of $\mathcal{B}(B^{-}\to K^{+}{\pi }^{-}{\pi }^{-})<9.5\times {10}^{-7}$ and $\mathcal{B}(B^{-}\to K^{-}{K}^{-}{\pi }^{+})<1.6\times {10}^{-7}$ on the corresponding branching fractions, including systematic uncertainties.

Figure 1

Example standard model decay diagrams for the decays $B^{-}\to K^{+}{\pi }^{-}{\pi }^{-}$ and $B^{-}\to K^{-}{K}^{-}{\pi }^{+}$, respectively.

Figure 2

Projections of the selected events with the fit results overlaid. The top (bottom) set of plots are for $B^{-}\to K^{+}{\pi }^{-}{\pi }^{-}$ ($B^{-}\to K^{-}{K}^{-}{\pi }^{+}$). From left to right the plots show the projections onto $m_{\mathrm{ES}}$, $\Delta E$, and the output of the neural network. The black points are the data, the solid blue curve is the total fit, the dotted red curve is the continuum background, the dashed green curve is the total background, and the dash-dotted black curve at the bottom is the signal. The continuum component has been suppressed in these plots by applying an additional requirement on the ratio of the signal likelihood to the sum of the signal and continuum likelihoods, calculated without use of the plotted variable. The value of the requirement for each plot is chosen to reject about 95% of the continuum background while retaining about 55% of the signal.

Figure 3

Construction of the confidence region bands. The left (right) plot is for $B^{-}\to K^{+}{\pi }^{-}{\pi }^{-}$ ($B^{-}\to K^{-}{K}^{-}{\pi }^{+}$). In each figure the blue dotted line shows the expected central value of $N_{\mathrm{fit}}$ as a function of the true branching fraction, the green solid (red dashed) lines show the 90% confidence level upper and lower limits including statistical and systematic errors (statistical errors only), the black dashed horizontal line marks the position of the previous upper limit [10], and the black dash-dotted lines indicate the results of this study.