Study of the $B^{-}\to J/\psi K^{-}{\pi }^{+}{\pi }^{-}$ decay and measurement of the $B^{-}\to X(3872)K^{-}$ branching fraction

We study the decay $B^{-}\to J/\psi K^{-}{\pi }^{+}{\pi }^{-}$ using $117\times {10}^6$ $B\overline{B}$ events collected at the $Y(4S)$ resonance with the BABAR detector at the PEP-II $e^{+}{e}^{-}$ asymmetric-energy storage ring. We measure the branching fractions $\mathcal{B}$ $(B^{-}\to J/\psi K^{-}{\pi }^{+}{\pi }^{-})=(116\pm 7(\mathrm{stat}.)\pm 9(\mathrm{syst}.))\times {10}^{-5}$ and $\mathcal{B}$ $(B^{-}\to X(3872){\mathrm{K}}^{-})\times$ $\mathcal{B}$ $(X(3872)\to \mathrm{J}/\psi {\pi }^{+}{\pi }^{-})=(1.28\pm 0.41)\times {10}^{-5}$ and find the mass of the $X(3872)$ to be $3873.4\pm 1.4\mathrm{MeV}/{\mathrm{c}}^2$. We search for the $h_c$ narrow state in the decay $B^{-}\to h_c{K}^{-}$, $h_c\to J/\psi {\pi }^{+}{\pi }^{-}$ and for the decay $B^{-}\to J/\psi D^0{\pi }^{-}$, with $D^0\to K^{-}{\pi }^{+}$. We set the $90\%$ C.L. limits $\mathcal{B}(B^{-}\to h_c{K}^{-})\times \mathcal{B}(h_c\to J/\psi {\pi }^{+}{\pi }^{-})<3.4\times {10}^{-6}$ and $\mathcal{B}(B^{-}\to J/\psi D^0{\pi }^{-})<5.2\times {10}^{-5}$.

Figure 1

Distribution of $m_{\mathrm{ES}}$ for (a) $B^{-}\to J/\psi K^{-}{\pi }^{+}{\pi }^{-}$ candidates, and (b) events in the $X(3872)$ region, $3862<m_{J/\psi \pi \pi }<3882\mathrm{MeV}/c^2$. The solid curves represent the binned likelihood fits described in the text; the combinatorial components are indicated by the dashed curves.

Figure 2

Distribution of $m_{J/\psi \pi \pi }$ (a) in the entire range, (b) in the $h_c$ region, (c) at the $\psi (2S)$, and (d)  in the region of the $X(3872)$ with the projection of the unbinned likelihood fit superimposed. The requirement ${\mathrm{m}}_{\mathrm{ES}}>5.27\mathrm{GeV}/c^2$ is applied.

Figure 3

Distribution of $m_{{\pi }^{+}{\pi }^{-}}$ (a) at the $X(3872)$ and (b) at the $\psi (2S)$, after subtraction of combinatorial and peaking background.

Figure 4

Distribution of $m{K}^{-}{\pi }^{+}$ in events $B^{-}\to J/\psi K^{-}{\pi }^{+}{\pi }^{-}$, with combinatorial background removed. Overlaid is an exponential fit. The arrow indicates the $3\sigma$ region expected for $D^0\to K^{-}{\pi }^{+}$.