Study of the decays $\psi(3686)\rightarrow\gamma\chi_{cJ}\rightarrow\gamma\bar{p}K^{*+}\Lambda+c.c.$ and $\psi(3686)\rightarrow\bar{p}K^{*+}\Lambda+c.c.$

Based on the data sample of $448.1\times10^{6}$ $\psi(3686)$ events collected with the BESIII detector at BEPCII, we present a study of the decays $\psi(3686)\rightarrow\gamma\chi_{cJ}\rightarrow\gamma\bar{p}K^{*+}\Lambda+c.c.$ and $\psi(3686)\rightarrow\bar{p}K^{*+}\Lambda+c.c.$. The branching fractions of $\chi_{cJ}\rightarrow\bar{p}K^{*+}\Lambda+c.c.$ ($J$=0, 1, 2) are measured to be $(4.8\pm0.7\pm0.5)\times10^{-4}$, $(5.0\pm0.5\pm0.4)\times10^{-4}$, and $(8.2\pm0.9\pm0.7)\times10^{-4}$, respectively, where the first uncertainties are statistical and the second systematic. The branching fraction of $\psi(3686)\rightarrow\bar{p}K^{*+}\Lambda+c.c.$ is measured to be $(6.3\pm0.5\pm0.5)\times10^{-5}$. All these decay modes are observed for the first time.


I. INTRODUCTION
The quark model provides a good description of both the ground states and some excited states of baryons. However, several resonances that are predicted by this model have not yet been observed, and hence there is an intense experimental effort underway to find these missing states [1]. The baryon coupling in conventional production channels (e.g. γ-nucleon) can be quite small, but the coupling between baryons and χ cJ decays via gg gluons could be larger (e.g. ψ or χ cJ decays). For this reason, charmonium decay is a promising process to study excited nucleons and hyperons [2].
The BES Collaboration has reported a study of J/ψ →pK + Λ + c.c. and ψ(3686) →pK + Λ + c.c. decays [3], in which a threshold enhancement in thē pΛ mass spectrum was observed. Throughout this paper, the inclusion of charge conjugate channels is implied. The BESIII Collaboration also reported a study of ψ(3686) → γpK + Λ [4], where a near threshold enhancement in the mass spectrum ofpΛ was observed in χ c0 decay. This enhancement may be interpreted as a quasibound dibaryon state, or as an enhancement due to final-state interaction, or simply as an interference effect of high-mass N * and Λ * states [4]. The study of the resonant structures in the similar decay modes ψ(3686) → γχ cJ → γpK * + Λ and ψ(3686) →pK * + Λ may help in the understanding of thepΛ threshold structure.

II. BESIII DETECTOR AND MONTE CARLO SIMULATION
The Beijing Electron Positron Collider II (BEPCII) is a double-ring e + e − collider running at center-of-mass energy ranging from 2.0 to 4.6 GeV. The BESIII detector [6] at BEPCII, with a geometrical acceptance of 93% of the 4π solid angle, operates in a magnetic filed of 1.0 T provided by a superconducting solenoid magnet. The detector is composed of a helium-based main drift chamber (MDC), a plastic-scintillator time-of-flight (TOF) system, a CsI(Tl) electromagnetic calorimeter (EMC) and a resistive plate chambers (RPC)-based muon chamber (MUC). The spatial resolution of the MDC is better than 130 µm, the charged track momentum resolution is 0.5% at 1 GeV/c, and the energy-loss (dE/dx) resolution is better than 6% for electrons from Bhabha events. The time resolution of the TOF is 80 ps (110 ps) in the barrel (endcaps. The energy resolution of the EMC at 1.0 GeV is 2.5% (5%) in the barrel (endcaps). The position resolution in the MUC is better than 2 cm.
Simulated Monte Carlo (MC) events are used to determine the detection efficiency, optimize selection criteria and estimate the level of contamination from background processes. The geant4-based [7] simulation package boost includes a geometric and material description of the BESIII detector, detector response, and digitization models, and also tracks the running conditions and performance of the detector. The production of ψ(3686) events is simulated with kkmc [8], where the known decay modes are generated by evtgen [9,10] with their BFs taken from the Particle Date Group (PDG) [11], and the remaining unknown decays are generated by lundcharm [12]. Exclusive MC samples of ψ(3686) → γχ cJ → γpK * + Λ and ψ(3686) →pK * + Λ are generated to determine detection efficiencies. In the signal MC simulation, the angular distribution of the decay ψ(3686) → γχ cJ has the form 1 + α cos 2 θ with α=1, −1/3, 1/13 for J =0, 1, 2, respectively, where θ is the photon polar angle [13]. The weak decay of Λ is generated with a model that includes parity violation. Other relevant decays are generated with besevtgen [10] with a uniform distribution in phase space.
The process ψ(3686) → γχ cJ → γpK * + Λ is reconstructed with Λ → pπ − , K * + → K + π 0 , and π 0 → γγ. Events are required to have at least two positive and two negative charged tracks. For each charged track, the polar angle in the MDC must satisfy | cos θ| < 0.93. The combined TOF and dE/dx information is used to form particle identification (PID) confidence levels for pion, kaon and proton hypotheses. Each track is assigned to the particle hypothesis with the highest confidence level. The identifiedp and K + candidates are further required to have their point of closest approach to the interaction point (IP) within ±1 cm in the plane perpendicular to beam direction and within ±10 cm in the plane of the beam direction. A common vertex constraint is applied to all pπ − pairs assumed to arise from a Λ decay, and the production of the Λ candidates is constrained to be at the interaction point. Only dE/dx information is used for the PID of p and π − candidates in Λ decays, because many of these particles do not reach the TOF on account of their low momentum.
Photon candidates are required to have energy deposition greater than 25 MeV in the barrel EMC (| cos θ| < 0.8) and 50 MeV in the end cap EMC (0.86 < | cos θ| < 0.92). To exclude showers from charged tracks, the angle between the direction of the photon and the nearest charged track is required to be greater than 5 • . In addition, the angle between the direction of the photon and anti-proton is required to be greater than 10 • to suppress background from anti-proton annihilation in the detector. The measured EMC time is required to be within 0 and 700 ns of start time of the event to suppress electronic noise and any energy deposition unrelated to the event.
To improve the mass resolution, the selected photons, anti-proton, kaon, and Λ candidate are subjected to a five-constraint (5C) kinematic fit under the hypothesis of ψ(3686) → γpK + π 0 Λ with the invariant mass of the two photons being constrained to the π 0 mass. The χ 2 of the 5C fit is required to be less than 70. For events with more than one combination satisfying this requirement, only the combination with the smallest χ 2 is accepted. To veto background events from ψ(3686) →pK + π 0 Λ and ψ(3686) → γpK + Λ, an alternative 5C (4C) kinematic fit is performed under the hypotheses of ψ(3686) →pK + π 0 Λ (γpK + Λ). We further require the confidence level of the kinematic fit for the ψ(3686) →pK + π 0 Λ assignment to be larger than those for the ψ(3686) → γpK + π 0 Λ and ψ(3686) → γpK + Λ hypotheses.
After applying these requirements, χ cJ signals are clearly seen in the invariant mass spectrum of pK * + Λ, as shown in Fig. 2. The mass windows used to select the χ c0 , χ c1 , χ c2 candidates correspond to about three times the χ cJ width convolved with the mass resolution, which are 3.35-3.48, 3.49-3.53, and 3.53-3.59 GeV/c 2 , respectively. The invariant  mass spectra of thepK * + ,pΛ, and K * + Λ combinations and the corresponding Dalitz plots are shown in Fig. 3 for each χ cJ state. No significant substructure is seen in the Dalitz plots ofpK * + Λ distributions. In order to search for the near-threshold structure of Mp Λ observed in Ref. [4] in the decay χ c0 →pK + Λ, fits are performed on Mp Λ where the structure is described by a weighted Breit-Wigner resonance with parameters fixed to those reported in Ref. [4]. These fits return a statistical significance for the structure of 2.1σ, 2.5σ, and 1.9σ for the χ c0 , χ c1 , and χ c2 states, respectively.

B. Background study
Using an inclusive MC sample of 506×10 6 ψ(3686) events, the background from fake Λ is found together with fake K * + . So, the background can be categorized into the following four types: (1) events with a genuine K * + and a fake χ cJ (K * , non-χ cJ ); (2) events with a genuine χ cJ and a fake K * (χ cJ , non-K * ); (3) events with fake K * and χ cJ candidates (non-K * , non-χ cJ ); (4) events containing a genuine K * + and a genuine χ cJ (K * , χ cJ ). The contributions from the first three categories can be estimated by performing a two-dimensional (2-D) fit to the distribution of M K + π 0 versus Mp K * + Λ . The fourth type of background events come mainly from the processes ψ(3686) → γχ cJ → γpK * + Λ → γγpK + Λ, The first two of these contributions are negligible, on account of the low BF of radiative K * + and Λ decays. The level of contamination coming from the other two modes is assessed by applying the selection to samples of exclusive MC events. For the normalization procedure, the BF of ψ(3686) → γχ cJ , χ cJ → γJ/ψ, J/ψ →pK * + Λ is estimated to be less than 10 −5 , which implies negligible background of less than one event from this source. The normalized number of ψ(3686) → γχ cJ , χ cJ →pK * + Σ 0 background events is estimated to be 11.7±3.5, 5.1±2.3, 4.8±2.6 for χ cJ (J=0, 1, 2), where the relative BFs used to calculate these yields are estimated from dedicated studies with the same data sample.
To investigate possible background from continuum processes, the same selection criteria are applied to a data sample of 2.93 fb −1 [14] collected at √ s = 3.773 GeV. After normalizing to the integrated luminosity of the ψ(3686) data sample, 20.1±4.1 events survive and no peak is found in the mass spectrum of Mp K * + Λ . As a cross check the selection is also performed on a data sample of 44.5 pb −1 collected at √ s = 3.65 GeV. Only one event survives, which corresponds to 14 events when normalized to the integrated luminosity of the ψ(3686) data sample, and is consistent with the result of the first study. In the BF measurement any continuum contribution is included in the other sources of non-peaking background and the total is estimated by the 2-D fit described below.
The shape of the K * + resonance, F K * sig , is described by a P -wave Breit-Wigner (BW) function [15] convolved with a double-Gaussian function (DG) that accounts for detector resolution, the parameters of which are determined from MC simulation. The definition of F K * sig is where Γ(s) = Γ( M s ) 2 ( q q0 ) 2L+1 , s is the invariant mass of the K + π 0 pair, M and Γ are the K * + mass and width [11], q is the K + momentum in the K * + rest frame, q 0 is the q value at s = M , and L = 1 is the relative orbital angular momentum of K + π 0 .
The background distribution of the fake K * + contribution, F non−K * bkg , is described by truncated polynomial function F non−K * bkg (s) = (s − m t ) a e −bs−cs 2 [15], where m t is the threshold mass for K + π 0 and a, b, c are free parameters.
The shape of the χ cJ signal is described by (3) Here E 3 γ is an E1 radiative-transition factor and f (E γ ) = E 2 0 Eγ E0+(Eγ −E0) 2 is a damping factor [16], where E γ is the energy of the radiative photon in the ψ(3686) rest frame and E 0 = In the relativistic BW function BW (m), the mass and width of the χ cJ are fixed to the PDG [11] values. The Blatt-Weisskopf barrier factor [17] B l (Q) is a function of Q, which is the momentum of either the radiative photon or the χ cJ in the ψ(3686) rest frame, Q 0 is the Q value at m = M χcJ , where m is the invariant mass of thepK * + Λ combination. Finally, G(m; µ, σ) is a modified Gaussian function parameterizing the instrumental mass resolution, taking the form [18] G(m; µ, σ) where the parameters are determined from MC simulation.
The shape of fake χ cJ candidates, F non−χcJ bkg , is described by an ARGUS [19] function.
The fit yields 254 ± 35 (K * + , χ c0 ) events with a statistical significance of 7.2σ, 328 ± 36 (K * + , χ c1 ) events with a statistical significance of 11.6σ, and 476 ± 52 (K * + , χ c2 ) events with a statistical significance of 15.2σ. The statistical significance is determined from the change of the log-likelihood value and the degrees of freedom in the fit when performed with and without a signal component. The 2-D histogram sampled from the composite PDF and the projections of the fit on the M K + π 0 and Mp K * + Λ distributions are shown in Fig. 4.

A. Event Selection
Events are selected containing at least two photons, onep, one K + , and one Λ candidate, iden-tified using the same criteria as employed in the ψ(3686) → γpK * + Λ analysis. The selected particles are subjected to a 5C kinematic fit under the hypothesis of ψ(3686) →pK + π 0 Λ, with the invariant mass of the two photons constrained to the π 0 mass. The χ 2 of the 5C fit is required to be less than 100. For events with more than one combination meeting this requirement, only the combination with the smallest χ 2 is retained for further analysis. To veto backgrounds from ψ(3686) → γpK + π 0 Λ and ψ(3686) → γpK + Λ, an alternative 5C (4C) kinematic fit is performed under the ψ(3686) → γpK + π 0 Λ (γpK + Λ) hypothesis. We further require that the confidence level of the kinematic fit for the ψ(3686) →pK + π 0 Λ assignment is larger than those of the ψ(3686) → γpK + π 0 Λ and ψ(3686) → γpK + Λ hypotheses.
The distribution of M K + π 0 versus M pπ − is shown in Fig. 5(a), where K * + and Λ signals are visible. The Λ candidates are selected by requiring |M pπ − − M Λ | < 6 MeV/c 2 and K * + candidates are selected by requiring |M K + π 0 − M K * + | < 0.1 GeV/c 2 . The K * + sidebands are defined to be 1.1 < M K + π 0 < 1.2 GeV/c 2 and 0.65 < M K + π 0 < 0.75 GeV/c 2 . The distribution of M pπ − for events within the K * + signal region is shown in Fig. 5(b). The mass spectra ofpK * + ,pΛ, K * + Λ, and Dalitz plot after the application of all selection criteria are shown in Fig. 6. A near-threshold structure in the Mp Λ is fitted with a 1.7σ signficance, using the the same parameterization as in the χ cJ →pK * + Λ analysis.

B. Background study
Using an inclusive MC sample of 506×10 6 ψ(3686) events, the background from fake Λ is found together with fake K * + . The sources of background can be categorized into two types: peaking background events with genuine K * + mesons in the final state and non-peaking background events with fake K * + candidates. The non-peaking background can be estimated from a fit to the M K + π 0 spectrum. The major peaking backgrounds are found to be: ψ(3686) → γχ cJ → γpK * + Λ (J=0, 1, 2) and ψ(3686) →pK * + Σ 0 , Σ 0 → γΛ. Corresponding exclusive MC samples are generated for further studies. The selection criteria are applied to these exclusive MC samples and the number of surviving events are normalized by the BFs of the relevant decay processes. The normalized number of ψ(3686) → pK * + Σ 0 background events is 5.2±1.1 and the expected numbers of ψ(3686) → γχ cJ → γpK * + Λ (J=0, 1, 2) background decays are 1.9±0.3, 4.5±0.5 and 8.8±1.0, respectively.
A data sample of 2.93 fb −1 [14] collected at √ s = 3.77 GeV is used to investigate possible background from continuum processes. After normalizing to the integrated luminosity of the ψ(3686) data sample, 164.1±9.5 events survive and a clear K * + peak is found in the K + π 0 mass spectrum. This background yield is cross-checked by repeating the procedure on the data sample of 44.5 pb −1 [20] collected at √ s = 3.65 GeV, and a compatible result of 207±61 events is obtained, after normalization.
C. Branching fraction measurement of ψ(3686) →pK * + Λ An unbinned maximum likelihood fit is performed to the distribution of M K + π 0 (Fig. 7) to extract the number of K * + signal events. The K * + sig-nal shape is described by a P -wave BW function convolved with a double-Gaussian function, and the background shape is described by a truncated polynomial function. The definitions of these functions are the same as those introduced in Sec. III C. The fit result is shown in Fig. 7.
The BF of ψ(3686) →pK * + Λ is calculated according to where N obs sig = 1011 ± 60 is number of K * + signal events obtained from the fit, N bkg = 20.4 ± 1.6 is the number of peaking background events reported in Sec. IV B, and ǫ is the detection efficiency, (14.0 ± 0.1)%, estimated from MC simulation. The B(ψ(3686) →pK * + Λ) is measured to be (6.3±0.5)×10 −5 , where the uncertainty is statistical only.

V. SYSTEMATIC UNCERTAINTIES
Systematic uncertainties on the BF measurements arise from a variety of sources:  Tracking efficiency. The uncertainty due to data-MC difference in the tracking efficiency is 1% for each charged track coming from a primary vertex according to a study of J/ψ → K * K and J/ψ → ppπ + π − events. For each track from Λ, the uncertainty is also 1% from analysis of J/ψ →pK + Λ events [4].
PID efficiency. The candidates require tracks to be identified as p,p, K + , or π − . The PID efficiency have been investigated using control samples of J/ψ → K 0 S K ± π ± and J/ψ → ppπ + π − [21,22]. The uncertainty is assigned to be 1% per charged track.
Photon detection efficiency. The photon detection efficiency was studied in the analysis of J/ψ → ρπ decays [21]. The difference in the detection efficiency between the data and MC simulation is taken as the systematic uncertainty from this source, and 1% is assigned for each photon.
Λ Mass window. The systematic uncertainty from the requirement on the Λ signal region is estimated by smearing the pπ − invariant mass in the signal MC sample with a Gaussian function to compensate for the resolution difference between data and MC simu- lation. The smearing parameters are determined by fitting the Λ distribution in data with the MC shape convolved with a Gaussian function. The difference in the detection efficiency as determined from signal MC sample with and without the extra smearing is taken as the systematic uncertainty. Kinematic fit. The systematic uncertainty due to kinematic fitting is estimated by correcting the helix parameters of charged tracks according the method described in Ref. [23]. The differences in the detection efficiency between the MC samples with and without this correction are taken as the uncertainties, which are 0.1%, 0.5%, and 0.2% for χ cJ → pK * + Λ (J=0, 1, 2) and 1.4% for ψ(3686) →pK * + Λ.
Fit range. To estimate the systematic uncertainty due to fit range, several alternative fits in different ranges are performed. The resulting largest difference in the BF is assigned as the systematic uncertainty.
Signal shape. To estimate the uncertainty due to the choice of signal shape, the K * + and χ cJ signal line shapes are replaced by alternative fits using MC shapes and the resulting differences in the BFs are assigned as systematic uncertainties.
Background shape.
In the measurements of B(χ cJ →pK * + Λ) and B(ψ(3686) →pK * + Λ), the χ cJ background shape is described by an ARGUS function and the K * + background shape is described by a second-order truncated polynomial function. To estimate the systematic uncertainty due to choice of background shape, an alternative fit is performed in which the ARGUS function is replaced with a second-order Chebychev polynomial function and the K * + signal is described with a third-order truncated polynomial. The change in the measured BF is assigned as the corresponding systematic uncertainty.
The above systematic uncertainties are summa-rized in Table I. The total systematic uncertainty is calculated by assuming the individual components to be independent, and adding their magnitude in quadrature.

ACKNOWLEDGMENTS
The BESIII collaboration thanks the staff of BEPCII and the IHEP computing center for their strong support. This work is supported in part by National Key