Measurement of branching fraction and final-state asymmetry for the $\bar{B}^{0}\to K^{0}_{S}K^{\mp}\pi^{\pm}$ decay

We report a measurement of the branching fraction and final-state asymmetry for the $\bar{B}^{0}\to K^{0}_{S}K^{\mp}\pi^{\pm}$ decays. The analysis is based on a data sample of 711 $\rm{fb}^{-1}$ collected at the $\Upsilon(4S)$ resonance with the Belle detector at the KEKB asymmetric-energy $e^{+}e^{-}$ collider. We obtain a branching fraction of $(3.60\pm0.33\pm0.15)\times10^{-6}$ and a final-state asymmetry of $(-8.5\pm8.9\pm0.2)\%$, where the first uncertainties are statistical and the second are systematic. Hints of peaking structures are seen in the differential branching fractions measured as functions of Dalitz variables.

We report a measurement of the branching fraction and final-state asymmetry for theB 0 → K 0 S K ∓ π AE decays. The analysis is based on a data sample of 711 fb −1 collected at the ϒð4SÞ resonance with the Belle detector at the KEKB asymmetric-energy e þ e − collider. We obtain a branching fraction of ð3.60 AE 0.33 AE 0.15Þ × 10 −6 and a final-state asymmetry of ð−8.5 AE 8.9 AE 0.2Þ%, where the first uncertainties are statistical and the second are systematic. Hints of peaking structures are found in the differential branching fractions measured as functions of Dalitz variables. Three-body charmless hadronic B decays are suppressed in the standard model (SM); the ones with an even number of kaons have a smaller decay rate compared to those with an odd number of kaons. These three-body decays proceed via b → u tree and W-exchange diagrams, as well as b → s; d penguin processes with a virtual loop. The latter provides an opportunity to search for physics beyond the SM since new heavy particles may cause deviations from SM predictions. Due to possible interference between the aforementioned diagrams, these decays are sensitive to CP violation localized in the Dalitz plane [1,2].
Previous measurements ofB 0 → K 0 S K ∓ π AE decays [3] by BABAR [4,5] and LHCb [6][7][8] found hints of structures in the low K − π þ and K − K 0 S mass regions with a highly asymmetric distribution in helicity angle. These studies also reported no two-body resonance decays withKK Ã final states. Furthermore, the yields were not sufficient to draw firm conclusions with a Dalitz plot analysis. Similar studies on B þ → K þ K − π þ performed by Belle [9], BABAR [10], and LHCb [11,12] found an unexpected peak in the K þ K − invariant mass (M K þ K − ) as well as strong evidence for localized CP violation near M K þ K − < 1.5 GeV=c 2 . Assuming the excess is due to a two-body resonance, a search for its isospin partner decaying to K − K 0 S would help elucidate the nature of the enigmatic resonance.
We report measurements of the branching fraction and final-state asymmetry ofB 0 → K 0 S K ∓ π AE decays. Using the charges of final-state particles, the latter is defined as where N denotes the signal yield obtained for the corresponding final state of both B 0 andB 0 . Here A is distinct from the direct CP asymmetry (A CP ); rather it is an asymmetry between the decay final states of K 0 K − π þ andK 0 K þ π − where the K 0 orK 0 is reconstructed as a K 0 S . We measure this quantity as it can be more precisely determined than A CP for this decay mode. A nonzero A value would be an indirect manifestation of CP violation. This is the first measurement of such an asymmetry in thē B 0 → K 0 S K ∓ π AE decay. In addition, we use the s Plot [13] method to obtain background-subtracted yields for the Dalitz variables M K − π þ , M π þ K 0 S , and M K − K 0 S , and hence to determine their differential branching fractions. The total branching fraction is calculated by integrating the differential branching fraction. By utilizing this well-established method, we can infer the existence of an intermediate resonance and localized asymmetry in the backgroundsubtracted Dalitz plot, as well as compare the result from this study with previous measurements [4][5][6][7][8][9][10][11][12].
Our measurement is based on the full data sample of 711 fb −1 , corresponding to 772 × 10 6 BB pairs, collected on the ϒð4SÞ resonance with the Belle detector [14] at the KEKB asymmetric-energy e þ e − collider [15]. The detector components relevant for this study are a silicon vertex detector (SVD), a 50-layer central drift chamber (CDC), an array of aerogel threshold Cherenkov counters (ACC), a barrel-like arrangement of time-of-flight scintillation counters (TOF), and an electromagnetic calorimeter made of CsI(Tl) crystals. These are located inside a superconducting solenoid that provides a 1.5 T magnetic field.
Large samples of Monte Carlo (MC) events are generated with EVTGEN [16] and subsequently simulated with GEANT3 [17] with the configurations of the Belle detector. We use these samples to obtain the expected distributions of various physical quantities for signal and backgrounds, to optimize the selection criteria, and to determine the signal detection efficiency.
The selection criteria for the final-state particles in thē B 0 → K 0 S K ∓ π AE reconstruction are based on information obtained from the tracking systems (SVD and CDC) and the charged-hadron identification (PID) systems, namely the CDC, ACC, and TOF. The charged kaons and pions are required to have an impact parameter within AE0.2 cm of the interaction point (IP) in the transverse plane, and within AE5.0 cm along the z axis, defined as the direction opposite the e þ beam. The likelihood values of each track for kaon and pion hypotheses (L K and L π ) are determined from the information provided by the PID systems. A track is identified as a kaon if L K =ðL K þ L π Þ > 0.6, otherwise it is treated as a pion. The efficiency for identifying a pion (kaon) is about 88% (86%), depending on the track momentum, while the probability for a pion or a kaon to be misidentified is less than 10%. The efficiency and misidentification probabilities are averaged over the momentum of final-state particles.
The K 0 S candidates are reconstructed via the K 0 S → π þ π − decay, and the identification is enhanced by selecting on the output of a neural network (NN) [18], which combines seven kinematic variables of the K 0 S [19]. The invariant mass of the K 0 S candidate is required to be within AE10 MeV=c 2 of its world average [20], which corresponds to about three times the mass resolution. The K 0 S → π þ π − vertex fit is required to converge with a goodness-of-fit (χ 2 ) value less than 20.
We identify B mesons with two kinematic variables calculated in the center-of-mass (CM) frame: the beam- , and the energy difference ΔE ≡ E B − E beam , where E beam is the beam energy, and ⃗p B (E B ) is the momentum (energy) of the reconstructed B meson. The B candidates are required to have M bc > 5.255 GeV=c 2 and jΔEj < 0.15 GeV, with the signal region given by 5.272 GeV=c 2 < M bc < 5.288 GeV=c 2 and jΔEj < 0.05 GeV. We require a successful vertex fit forB 0 → K 0 S K ∓ π AE candidates, where the K 0 S trajectory is included in the fit, with χ 2 < 100. About 9% of events in the data sample have more than one B candidate, and the signal events have an average of 1.1 candidates. In such cases, we select the candidate with the smallest χ 2 value from the B vertex fit. According to simulations, our best candidate selection method chooses the correct candidate in 99% of cases.
The dominant background is from the continuum e þ e − → qqðq ¼ u; d; s; cÞ process. To suppress it, we construct a Fisher discriminant [21] from 17 modified Fox-Wolfram moments [22]. To further improve the distinguishing power, we combine the discriminant output with four more variables in an NN. These are the cosine of the angle between the reconstructed B flight direction and the z axis in the CM frame, the offset along the z axis between the vertex of the reconstructed B and the vertex formed by the remaining tracks, the cosine of the angle between the thrust axis [23] of the reconstructed B and that of the rest of the event in the CM frame, and a B meson flavor tagging quality variable. The NN is trained with signal and continuum MC samples. The NN output (C NN ) ranges from −1 to 1, and it is required to be greater than 0.7. This removes 93% of the continuum background while retaining 82% of the signal. We transform C NN to Background events from B decays mediated via the b → c transition (generic B decays) have peaking structures in the signal region. They are mainly due to the decays with two-body final states of D and J=ψ mesons, e.g., These decays can be identified by peaks at the nominal D and J=ψ mass [20] in the distributions of the invariant masses of two of the final-state particles where we allow for a change in the mass hypothesis of a charged kaon or pion). We exclude events within AE4σ of the nominal mass of the peaking structures to suppress the contributions from D and J=ψ mesons.
The rare B background from b → u; d; s transitions is studied with a large MC sample in which the branching fractions are much larger than the measured or expected values. Two modes are found to have peaks near the ΔE signal region: The signal yield and A are extracted from a threedimensional extended unbinned maximum likelihood fit, with the likelihood defined as where N is the number of candidate events, N j is the number of events in category j, i is the event index, q i is the charge of the K AE in the i-th event, A j is the value of final-state asymmetry of the j-th category, P j represents the value of the corresponding three-dimensional probability density function (PDF), and M i bc , ΔE i , and C 0i NN are the M bc , ΔE, and C 0 NN values of the ith event, respectively.
With all the selection criteria applied, 98% of the signal MC events are correctly reconstructed while 2% are selfcrossfeed (scf). In the fit, the ratio of scf to correctly reconstructed ("true") signal events is fixed. The signal yield (N sig ) is the combined yield of the true signal and scf PDF. Five more event categories are included in the fit: continuum background, generic B background, , and the remaining rare B background. The true signal PDF is described by the product of a sum of two Gaussian functions in M bc , a sum of three Gaussian functions in ΔE, and an asymmetric Gaussian function in C 0 NN . These PDF shapes are calibrated including possible data-MC differences obtained from the study of a high-statistics control mode B 0 → D − π þ , D − → K 0 S π − . The continuum background PDF is given by the product of an ARGUS function [24] in M bc , a secondorder polynomial in ΔE, and a sum of a Gaussian and an asymmetric Gaussian function in C 0 NN . All shape parameters of the continuum PDF are free in the fit, except for the ARGUS endpoint which is fixed to 5.2892 GeV=c 2 . For the other contributions (scf, generic B, S , and rare B), their PDFs are described by a smoothed histogram in ΔE and M bc , and an asymmetric Gaussian function in C 0 NN whose shape is based on MC. The yield of each category is floated. The A value is fixed to zero for all background categories.
The signal-enhanced projections of the fit are shown in Fig. 1. We obtain a signal yield of 490 þ46 −45 with a statistical significance of 13 standard deviations, and an A value of ð−8.5 AE 8.9Þ%. The significance is defined as ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi −2 lnðL 0 =L max Þ p , where L 0 and L max are the likelihood values obtained by the fit with and without the signal yield fixed to zero, respectively.
The branching fraction is calculated using where N sig , N BB , ϵ, and η are the fitted signal yield, the number of BB events (¼ 772 × 10 6 ), the signal reconstruction efficiency, and the efficiency calibration factor, respectively. We assume that charged and neutral BB events are produced equally at the ϒð4SÞ. The ϵ value  determined by MC, with all the selection criteria applied, is ð26.7 AE 0.1Þ%. The calibration factor includes contributions due to various systematic effects η ¼ η K × η π × η NN × η fit , where η K ð¼ 0.9948 AE 0.0083Þ and η π ð¼ 0.9512 AE 0.0079Þ are the corrections due to K AE and π AE identification, and are obtained from a control sample of D Ãþ → D 0 π þ , D 0 → K − π þ . Similarly, η NN ð¼ 0.9897 AE 0.0208Þ is due to the requirement on C NN and is obtained from a sample of The factor η fit ð¼ 1.022 AE 0.004Þ is due to fit bias, obtained from an ensemble test on the fitter. We also obtain backgroundsubtracted distributions after separating into five bins, and then calculate the differential branching fractions as functions of the three Dalitz variables with the yield and reconstruction efficiency within each bin. We use a similar binning scheme as the one in Ref. [9]. Figure 3 shows the  differential branching fractions as functions of the three Dalitz variables including comparison to the MC with a three-body phase space model. Large deviations from phase space expectations are found around 1.2 GeV=c 2 in the M K − K 0 S spectrum as well as near 3.0-4.2 GeV=c 2 in the M π þ K 0 S spectrum. No obvious structure is observed in the low-mass regions of both M K − π þ and M π þ K 0 S , which is consistent with previous two-body decay measurements [5,7,8].
Differential branching fractions are shown separately for the K 0 S K − π þ and K 0 S K þ π − final states in Fig. 4. Within each bin of the Dalitz variables, the results are consistent with no asymmetry. The details of the differential branching fraction calculation in each bin are summarized in Table I.
Various sources of systematic uncertainties in the branching fraction calculation are listed in Table II. The uncertainty due to the number of BB events is 1.4%. The uncertainty due to the charged-track reconstruction efficiency is estimated to be 0.35% per track by using partially reconstructed D Ãþ → D 0 π þ , D 0 → π þ π − K 0 S . The uncertainties due to K AE and π AE identification are obtained by the control sample study of D Ãþ → D 0 π þ , D 0 → K − π þ . The uncertainty due to the K 0 S → π þ π − branching fraction is based on its world average [20]. The uncertainty due to K 0 S identification is estimated to be 1.6% based on a D Ãþ → D 0 π þ , D 0 → K 0 S π 0 control sample [25]. The uncertainty due to continuum suppression with the requirement on C NN is obtained from a control sample of B 0 → D − π þ , D − → K 0 S π − . The uncertainty of the reconstruction efficiency is due to limited MC statistics. The uncertainty due to fixed PDF shapes is estimated by the deviation of fitted signal yield when varying the parameters of the PDFs in different cases. For all the smoothed histograms, we vary their binning parameters. For the PDFs with fixed parametrization, the fixed parameters are randomized by using a Gaussian random number to repeat data fits, and the uncertainty of the yield distribution is quoted. The uncertainty due to fit bias is obtained from an ensemble test on the fitter. Various sources of systematic uncertainties in A are listed in Table III. The uncertainty due to K AE and π AE TABLE I. Signal yields, efficiency, and differential branching fraction in each M K − π þ , M K − K 0 S , and M π þ K 0 S bin. detection bias are obtained from the control samples of D þ → ϕπ þ and D þ s → ϕπ þ [26], and D þ → K 0 S π þ [27], respectively. The uncertainties due to the fixed PDF shapes are treated similarly to those on the branching fraction. They are estimated from the deviation of the fitted value of A with varying the conditions of those PDFs in different cases.
In summary, we have measured the branching fraction and asymmetry A ofB 0 → K 0 S K ∓ π AE using a data sample of 711 fb −1 collected by Belle. We obtain a branching fraction of ð3.60 AE 0.33 AE 0.15Þ × 10 −6 and an A of ð−8.5 AE 8.9 AE 0.2Þ%, where the first uncertainty is statistical and the second is systematic. The measured A value is consistent with no asymmetry. Hints of peaking structures are seen in the regions M 2 K − K 0 S < 2 GeV 2 =c 4 and 7 GeV 2 =c 4 < M 2 π þ K 0 S < 23 GeV 2 =c 4 in the Dalitz plot.
The peaking structure in M 2 is consistent with the result from the previous B þ → K þ K − π þ measurement. A cross-check is performed by calculating the differential branching fraction after projecting onto each Dalitz variable, and hints of peaking structures are found near 1.2 GeV=c 2 in M K − K 0 S and around 4.2 GeV=c 2 in M π þ K 0 S when compared to the phase space MC. No obvious K Ã structure is seen in either low M K − π þ or M π þ K 0 S spectra, which are consistent with the BABAR and LHCb results [5,7,8]. No localized final-state asymmetry is observed. In the near future, experiments with large data sets such as Belle II and LHCb can perform a more detailed analysis exploiting the full Dalitz plot.