Measurement of charmless $B$ decays to $\eta K^{*}$ and $\eta \rho$

We report measurements of branching fractions and $CP$ asymmetries for $B\to \eta K^{*}$ and $B\to \eta \rho$ decays. These results are obtained from a $414{\mathrm{fb}}^{-1}$ data sample collected at the ${\rm Y}(4S)$ resonance with the Belle detector at the KEKB asymmetric-energy $e^{+}{e}^{-}$ collider. We measure the following branching fractions: $\mathcal{B}(B^0\to \eta K^{*0})=(15.2\pm 1.2\pm 1.0)\times {10}^{-6}$ and $\mathcal{B}(B^{+}\to \eta K^{*+})=({19.3}_{-1.9}^{+2.0}\pm 1.5)\times {10}^{-6}$, where the first error is statistical and the second systematic. We also find a $2.7\sigma$ excess in the $B^{+}\to \eta {\rho }^{+}$ mode and measure $\mathcal{B}(B^{+}\to \eta {\rho }^{+})=({4.1}_{-1.3}^{+1.4}\pm 0.4)\times {10}^{-6}<6.5\times {10}^{-6}$ at 90% confidence level. For $B^0\to \eta {\rho }^0$ decays, we determine the upper limit $\mathcal{B}(B^0\to \eta {\rho }^0)<1.9\times {10}^{-6}$ at 90% confidence level. The partial rate asymmetries are ${\mathcal{A}}_{CP}(\eta K^{*0})=0.17\pm 0.08\pm 0.01$, ${\mathcal{A}}_{CP}(\eta K^{*+})=0.03\pm 0.10\pm 0.01$, and ${\mathcal{A}}_{CP}(\eta {\rho }^{+})=-{0.04}_{-0.32}^{+0.34}\pm 0.01$.

Figure 1

Feynman diagrams for $B^{+}\to \eta K^{*+}$ and $\eta {\rho }^{+}$ decays. The corresponding neutral decays are similar except that the spectator quark becomes a $d$ and (b) and (c) diagrams do not exist.

Figure 2

Projections on $M_{\mathrm{bc}}$ (for the signal slice in $\Delta E$) and $\Delta E$ (for the signal slice in $M_{\mathrm{bc}}$) for $\eta K^{*0}$ (a,b) and $\eta K^{*+}$ (c,d) with the expected signal and background curves overlaid. The shaded area represents $\eta \to {\pi }^{+}{\pi }^{-}{\pi }^0$ decays.

Figure 3

Projections on $M_{bc}$ (for the signal slice in $\Delta E$) and $\Delta E$ (for the signal slice in $M_{\mathrm{bc}}$) from 2D ML fit results for $\eta {\rho }^0$ (a,b) and $\eta {\rho }^{+}$ (c,d) with the expected signal and background curves overlaid. The shaded area represents $\eta \to {\pi }^{+}{\pi }^{-}{\pi }^0$ decays.

Figure 4

Distributions of the $K^{*}$ helicity for the (a) $K^{*0}$ and (b) $K^{*+}$ modes in case of $B\to \eta K^{*}$ decays. The overlaid histograms represent the distributions from MC normalized by the 2D fit results.

Figure 5

Fitted yields vs the $K\pi$ invariant mass for the (a) $(K\pi {)}^0$ and (b) $(K\pi {)}^{+}$ modes. The overlaid functions are the results of a binned ${\chi }^2$ fit. The dashed line represents the contribution from the $D$-wave $K_2^{*}(1430)$, and the dashed-dotted line represents the LASS $S$-wave parameterization. LASS parameterization parameters, widths of $P$-wave and $D$-wave functions are allowed to float in the fits.

Figure 6

Fitted yields vs (a) ${\rho }^{+}$ helicity and (b) ${\pi }^{+}{\pi }^0$ invariant mass from $B\to \eta {\rho }^{+}$ decays. The overlaid histograms are expected distributions from MC and normalized by the 2-D fit results with ${\chi }^2/N=6.8/8$ for (a) and ${\chi }^2/N=13.6/11$ for (b).