Search for a new pseudoscalar decaying into a pair of muons in events with a top-quark pair at $\sqrt{s} = 13$ TeV with the ATLAS detector

A search for a new pseudoscalar $a$-boson produced in events with a top-quark pair, where the $a$-boson decays into a pair of muons, is performed using $\sqrt{s} = 13$ TeV $pp$ collision data collected with the ATLAS detector at the LHC, corresponding to an integrated luminosity of $139\, \mathrm{fb}^{-1}$. The search targets the final state where only one top quark decays to an electron or muon, resulting in a signature with three leptons $e\mu\mu$ and $\mu\mu\mu$. No significant excess of events above the Standard Model expectation is observed and upper limits are set on two signal models: $pp \rightarrow t\bar{t}a$ and $pp \rightarrow t\bar{t}$ with $t \rightarrow H^\pm b$, $H^\pm \rightarrow W^\pm a$, where $a\rightarrow\mu\mu$, in the mass ranges $15$ GeV $


Introduction
This paper describes a search for the production of a new light pseudoscalar particle decaying into a pair of muons in events with a top-quark pair.Such new particles are well motivated phenomenologically and have been proposed as an explanation for the excess of -ray emissions from the center of our galaxy [1][2][3][4] in the context of Coy Dark Matter models [5][6][7].They can be colorless solutions of the naturalness problem [8][9][10][11].A light scalar can also render the electroweak phase transition strong first-order, which is one of the ingredients for electroweak baryogenesis [12][13][14].These new particles are present in several extensions of the Standard Model (SM), where new light pseudoscalars mix with fields in an extended Higgs sector, which may include additional heavy neutral and charged scalars, inheriting the Yukawa couplings to fermions.In this case, the large coupling to top quarks suggests a search for this new light pseudoscalar produced in events with a top-quark pair [15].Two scenarios are considered: one where the new light particle  is produced in association with a top-quark pair ( t,  → ) and another where a top quark decays into a new charged Higgs boson that subsequently decays into a new light particle and a  boson ( →  + ,  + →  + ,  → ).The search focuses on the mass ranges 15 GeV <   < 72 GeV and 120 GeV ≤   ± ≤ 160 GeV.The high mass resolution achievable for muon pairs provides a distinctive signature to search for and excellent discrimination against most of the background sources.The search targets final states with three leptons, including an electron or muon from a top-quark decay in addition to the two muons from the light pseudoscalar decay.Figure 1 shows representative Feynman diagrams for the signal processes targeted by this search.This analysis uses data from proton-proton ( ) collisions at a center-of-mass energy of √  = 13 TeV collected by the ATLAS experiment during Run 2 (2015 to 2018) of the Large Hadron Collider (LHC), corresponding to an integrated luminosity of 139 fb −1 .Previous results include those from a more general search for multilepton signatures by the CMS Collaboration that sets upper limits of 1-10 fb on the production cross section for  t,  →  at 95% confidence level in the mass ranges   = 15-75 GeV and 108-340 GeV [16].The CMS Collaboration also performed a search for  ± →  + ,  →  in  →  ±  decays targeting the mass ranges   ± = 120-160 GeV and   = 15-75 GeV, and set upper limits of (1.9−8.6)× 10 −6 on the branching ratio B( →  + ,  + →  + ,  → ) at 95% confidence level [17].Previously, the CDF Collaboration searched for  ± →  ,  →  in  →  ±  decays, targeting the mass ranges   ± = 90-160 GeV and   = 4-9 GeV [18].

ATLAS detector
The ATLAS experiment [19] at the LHC is a multipurpose particle detector with a forward-backward symmetric cylindrical geometry and a near 4 coverage in solid angle. 1 It consists of an inner tracking detector (ID) surrounded by a thin superconducting solenoid providing a 2 T axial magnetic field, electromagnetic (EM) and hadron calorimeters, and a muon spectrometer.The inner tracking detector covers the pseudorapidity range || < 2.5.It consists of silicon pixel, silicon microstrip, and transition radiation tracking detectors.Lead/liquid-argon (LAr) sampling calorimeters provide EM energy measurements with high granularity.A steel/scintillator-tile hadron calorimeter covers the central pseudorapidity range (|| < 1.7).The endcap and forward regions are instrumented with LAr calorimeters for both the EM and hadronic energy measurements up to || = 4.9.The muon spectrometer surrounds the calorimeters and is based on three large superconducting air-core toroidal magnets with eight coils each.The field integral of the toroids ranges between 2.0 and 6.0 T m across most of the detector.The muon spectrometer includes a system of precision chambers for tracking and fast detectors for triggering.A two-level trigger system is used to select events.The first-level trigger is implemented in hardware and uses a subset of the detector information to accept events at a rate below 100 kHz.This is followed by a software-based trigger that reduces the accepted event rate to 1 kHz on average depending on the data-taking conditions.An extensive software suite [20] is used in data simulation, in the reconstruction and analysis of real and simulated data, in detector operations, and in the trigger and data acquisition systems of the experiment.

Data and simulated samples
The analysis uses   collision data collected with the ATLAS detector in Run 2 of the LHC, at a center-ofmass energy of √  = 13 TeV, corresponding to a total integrated luminosity of 139 fb −1 [21].Events were recorded using single-lepton triggers with either a low  T threshold and a lepton isolation requirement, or a higher threshold but a looser identification criterion and without any isolation requirement.The lowest  T threshold in the single-muon trigger was 20 (26) GeV [22] for data taken in 2015 (2016-2018), while in the single-electron trigger it was 24 (26) GeV [23].Only events for which the LHC beams were in stable-collision mode and all relevant subsystems were operational are used [24].Simulated signal and background samples are used to describe background sources, to estimate signal efficiency and acceptance, and to design and optimize the analysis.Monte Carlo (MC) samples were produced using either the full ATLAS detector simulation [25] based on Geant4 [26] or a faster simulation where the full Geant4 simulation of the calorimeter response is replaced by a detailed parameterization of the shower shapes [25].The effects of multiple inelastic interactions in the same and neighboring bunch crossings (pileup) were modeled by overlaying each simulated hard-scattering event with inelastic   events generated with Pythia 8.186 [27] using the NNPDF2.3loset of parton distribution functions (PDFs) [28] and the A3 set of tuned parameters (tune) [29].Simulated events are reweighted to match the pileup conditions observed in the full Run 2 dataset.All simulated events are processed through the same reconstruction algorithms and analysis chain as the data.Table 1 summarizes all the generated samples. 1 ATLAS uses a right-handed coordinate system with its origin at the nominal interaction point (IP) in the center of the detector and the -axis along the beam pipe.The -axis points from the IP to the center of the LHC ring, and the -axis points upwards.Cylindrical coordinates (, ) are used in the transverse plane,  being the azimuthal angle around the -axis.
The pseudorapidity is defined in terms of the polar angle  as  = − ln tan(/2).Angular distance is measured in units of The precision of the matrix element (ME) generators is next-to-leading order (NLO) in quantum chromodynamics (QCD) for most samples.In all samples where the parton shower (PS), hadronization, and multiparton interactions (MPI) were generated with Pythia 8, the decays of -and -hadrons were simulated using the EvtGen 1.6.0program [30], and the A14 tune [31] and the NNPDF2.3loPDF set [28] were used.Several samples were simulated with the Sherpa 2.2 generator [32].In this setup, NLO-accurate MEs for up to two partons, and MEs with leading-order (LO) accuracy for up to four partons, were calculated with the Comix [33] and OpenLoops libraries.They were matched with the Sherpa parton shower [34] by using the MEPS@NLO prescription [35][36][37][38] with the tune developed by the Sherpa authors and based on the NNPDF3.0nnloset of PDFs [39].The top-quark mass was set to   = 172.5 GeV.The  sample includes the contribution of  * starting at  ℓℓ > 5 GeV.The overlap between the  ( t) and the + jets ( t) samples is removed following the prescription in Ref. [40].
The  t signal samples were simulated with the MadGraph5_aMC@NLO 2.7.3 generator [51] at NLO precision.The FeynRules [52,53] model used is based on Ref. [54] and it assumes a pseudoscalar coupling between the -boson and fermions.The simulated signal samples use the NNPDF3.0nloPDF set.Tau-lepton decays of the top-quark pair are explicitly included even though their acceptance is relatively small.The production includes -, -and  q-initiated processes.Diagrams with a single top quark, such as   and  are not included.The leading-order  production process is shown in Figure 1.
In the case of  q-initiated production, three-body decays  →   are considered via a Breit-Wigner approximation in the whole mass spectrum of this search.Decays of the top quarks are simulated using the MadSpin program [55] by setting the parameter bwcut to at least (  −   )/Γ  , where Γ  is the top-quark width.The -boson is simulated with a narrow-width approximation.The events were interfaced to Pythia 8.210 [56], which used the A14 tune for modeling of showering and hadronization effects.The simulated mass points correspond to -boson masses of 12, 16, 20, 25, 30, 40, 50, 60, 70 and 77 GeV.
The  →  + ,  + →  +  signal samples were simulated using the MadGraph5_aMC@NLO 2.3.3 generator interfaced with Pythia 8.186, which used the A14 tune.This leading-order simulation uses the NNPDF2.3loPDF set.Both  + and  were simulated using a narrow-width assumption because the width is always smaller than the detector resolution for the investigated parameter space.The simulated signal samples include the CP-conjugate state of  + .The minor contributions from single-top-quark decays are neglected.A two-dimensional grid was chosen for the simulated ( + , ) masses from 15 GeV to 75 GeV in   and from 120 GeV to 160 GeV in   + .Mass points were generated in steps of 15 GeV for the -boson mass and 20 GeV for the  + boson mass.

Object and event selection
Electrons are reconstructed from tracks in the ID associated with topological clusters of energy deposits in the calorimeter [57] and are required to have  T > 27 GeV and || < 2.47.Electrons in the calorimeter barrel-endcap transition region (1.37 < || < 1.52) are excluded.Electrons must satisfy the "Medium" likelihood identification criterion and a very tight isolation requirement ("PLVTight") [57,58], which takes into account calorimeter energy deposits and charged-particle tracks (including the lepton track) in a cone around the lepton direction in order to reject electrons that likely originated from light-or heavy-flavor hadrons.Electron tracks must match the primary vertex of the event,2 i.e. they have to satisfy | 0 sin()| < 0.5 mm and , where  0 is the longitudinal impact parameter relative to the primary vertex and  0 (with uncertainty ( 0 )) is the transverse impact parameter relative to the beam line.
Muon candidates are identified by matching ID tracks to full tracks or track segments reconstructed in the muon spectrometer, using the "Medium" identification criterion [60].Muons are required to have  T > 10 GeV and || < 2.5.Muon tracks must also satisfy | 0 sin()| < 0.5 mm to reject pileup tracks.A "tight" muon selection is defined by additionally requiring that the muon candidate satisfies | sig 0 | < 3 and the isolation criterion "PFlowLoose" with   T -dependent Δ cone radius [60].The isolation criterion is corrected for the presence of nearby muons by subtracting from the isolation sum the  T of a track associated with any other loose muon within the isolation cone of the muon.This correction is particularly important at low   , where the angular separation of the two muons from the -boson decay can be small, less than the cone radius in the isolation calculation.All leptons are required to satisfy Δ(ℓ 1 , ℓ 2 ) > 0.1 to reject decay chains of hadrons that produce multiple leptons.
Jets are reconstructed from tracks in the ID and topological energy clusters in the calorimeter [61].Tracks matched to energy clusters, as well as unmatched energy clusters, are used in a particle-flow algorithm which determines the inputs for an anti-  clustering algorithm [Cacciari:2008gp] with a radius parameter of 0.4.Jets are required to satisfy  T > 20 GeV and || < 2.5.The effect of pileup is reduced by an algorithm which uses tracking information to require that the calorimeter-based jets are consistent with originating from the primary vertex [62].Jets containing -flavored hadrons ("-jets") are identified by a multivariate discriminant "DL1r" [63][64][65] combining track impact parameter values with information from secondary vertices reconstructed within the jet using a deep feed-forward neural network.A working point corresponding to 70% efficiency for identifying -jets and rejection rates of 9.4 for -jets and 390 for light-flavor jets measured with simulated  t events is used.
An overlap removal procedure is applied to prevent double-counting of objects.The closest jet within Δ  = √︁ (Δ) 2 + (Δ) 2 = 0.2 of a selected electron is removed, where  = [( +   )/( −   )]/2.If the nearest jet surviving that selection is within Δ  = 0.4 of an electron, the electron is discarded.Muons are usually removed if they are separated from the nearest jet by Δ  < 0.4, since this reduces the background from heavy-flavor decays inside jets.However, if this jet has fewer than three associated tracks, the muon is kept and the jet is removed instead; this avoids an inefficiency for high-energy muons undergoing significant energy loss in the calorimeter.Electrons are removed if they share their track with a muon.
Events are required to have at least one primary vertex with two or more tracks with  T > 0.5 GeV [59] and to have either exactly one electron and two opposite-charge muons, or three muons with total charge ±1.Events with electrons are selected with a single-electron trigger and the selected electron must match the object used in the trigger decision.Events with three muons are selected with a single-muon trigger.
Additional selections for muon candidates are used depending on the value of the -boson mass being considered.The two opposite-charge muons with a reconstructed invariant mass closest to the mass of the hypothesized -boson are referred to as "-muons".The leading -muon is required to satisfy  T > 15 GeV.In events with three muons, the additional muon is interpreted as coming from a top-quark decay and is referred to as the "top-muon".The top-muon is required to satisfy  T > 27 GeV and match the trigger object used in the decision.Studies with simulated signal samples show that the -muons are correctly matched to muons from the -boson decay in more than 98% of the events.All muons in simulation are required to be matched to true muons originating from the prompt decays of vector bosons or -bosons.

Analysis strategy
Events are categorized into mutually exclusive regions defined by the number of muons satisfying the tight identification selection, the number of jets, the number of -jets and the invariant mass of muon pairs.The "signal regions" (SR) in both the  and the  final states are defined by 12 GeV <     < 77 GeV, where     is the mass of the two -muons.The range is chosen so as to minimize contamination from processes with Υ →  decays with the lower bound, and from processes with on-shell  →  decays with the upper bound.In order to target events where only one of the top quarks decays to a lepton, the number of jets is required to be at least three and the number of -jets to be at least one.In the  final state, where there are two possible opposite-sign muon pairs, the mass of the "other" pair (not the pair chosen as -muons),  other  , is required to be  other  < 77 GeV or  other  > 107 GeV in order to reject events with  bosons.The SR regions are further binned in     .The width of the bins is chosen to be twice the expected     resolution, defined by the Gaussian core's width in the signal model.Several "control regions" (CR) are defined in order to target different sources of background.An "on-" control region is defined by requiring either 77 GeV <     < 107 GeV or 77 GeV <  other  < 107 GeV but otherwise the same minimum number of jets and -jets as in the SR.Two on- CRs are defined, one in the  final state and another one in the  final state.These regions are further divided into events with 3 jets and 1 -jet, events with 4 jets and 1 -jet, and events with at least 4 jets and 2 -jets.The regions with a small number of jets are enriched in   events, while the bins with a large number of jets are enriched in  t events.
Another CR, defined only in the  final state, targets  t processes where both top quarks decay leptonically.
In this case, one of the two muons is often produced in the nonprompt decay of a heavy-flavor hadron.This muon, called a "fake muon", is defined as the one with the same electric charge as the electron.Since both top quarks decay leptonically, events in this CR are required to have one or two jets, with exactly one of them being a -jet.This region is further binned in  ,fake T .
In all regions described above, all muons are required to satisfy the tight selection presented in Section 4.
Events with exactly one of the selected muons failing to satisfy the tight selection criteria are assigned to one of three "isolation sidebands" enriched in events from background processes with nonprompt muons.The three isolation sidebands correspond to different  T ranges (10 GeV ≤  T < 20 GeV, 20 GeV ≤  T < 40 GeV, and  T ≥ 40 GeV) of the muon which does not satisfy the tight selection criteria.The binning of the isolation sidebands is the same as in the corresponding signal and control regions.The regions used in the analysis are summarized in Table 2.

Background estimation
Several SM processes can produce final states which satisfy the object and event selections described above.The contributions from these processes are estimated with simulation or data-driven methods.
Background processes with all leptons originating from the prompt decay of vector bosons are described by simulation.The dominant source of background with prompt leptons is  t, the associated production of a  boson and a  t pair where the off-shell  boson decays into a low-mass muon pair.The normalization of this process is determined by data and largely constrained by the on- control region.The same  t normalization is used for all analysis channels.The categorization of the on- CR in number of jets and -jets allows a precise data-driven determination of the  t process normalization despite the large contamination from   events, which are subleading in the SR, as shown in Figure 2. Additional subleading sources of background in the SR include single top quarks produced in association with a  boson ( and  ), as well as  t,  t, triboson , and  t t processes.The normalization of each background process with prompt leptons other than  t is determined from simulation.Background processes with at least one lepton originating from the nonprompt decay of a hadron, from photon conversion, or from the misidentification of other particles, are described by simulation if the nonprompt lepton is an electron or by a data-driven method if the nonprompt lepton is a muon.Background processes with nonprompt electrons are greatly suppressed by the PLVTight isolation requirement.Nonprompt electrons are further classified depending on whether they originate from photon conversions or from hadrons.Background processes with nonprompt muons provide the largest number of events in the signal regions and the data-driven estimate ensures an accurate description.Simulation studies show that the contribution from processes with more than one nonprompt lepton is negligible.
As described in Section 5, each channel of the analysis is accompanied by three isolation sidebands enriched in events with nonprompt muons.The three sidebands are distinguished by the  T of the muon failing the tight selection criteria (10 GeV ≤  T < 20 GeV, 20 GeV ≤  T < 40 GeV, and  T > 40 GeV).The number of events from processes with nonprompt muons in the isolation sidebands is determined by data.The number of events from processes with nonprompt muons in signal and control regions is assumed to be proportional to the number of events in the three sidebands.The proportionality constant, called the fake factor, is different for each of the three sidebands, but otherwise common to all signal and control regions.The fake factors are determined in data and are largely constrained by the  t control region, which is divided into the same three bins of  ,fake T .Simulation studies show that background events with nonprompt muons arise mostly (≈ 90%) from  t processes where both top quarks decay leptonically.Simulation studies also show that nearly all other background events with nonprompt muons come from processes with semileptonic decays of heavy-flavor hadrons, akin to  t events.The number of different mass hypotheses simulated is limited by the computational resources available.The gap between simulated mass hypotheses is, however, much wider than the dimuon mass resolution of the ATLAS detector.
A parameterized model for the    spectrum, where    is the mass of the two -muons in each event, is used to probe -boson mass hypotheses between the values chosen for the simulated samples.The model uses the double Crystal Ball (dCB) probability density function [66].The parameters of the dCB distribution are evaluated independently for each simulated -boson mass using a maximum-likelihood fit.Only the Gaussian core's mean and width are observed to vary significantly, while the parameters describing the power-law tails are consistent for all -boson masses in the range considered in this search.The mean and the width of the Gaussian core vary linearly with the value of the -boson mass.The signal acceptance times efficiency is also observed to vary linearly with the value of the -boson mass.The dCB distribution parameters, their linear dependency on the -boson mass, and the model of the signal acceptance times efficiency are determined separately in the  and  channels and for the two signal models  t and  →  + ,  + →  +  in order to account for the different muon kinematics in the two models.Figure 4 shows the dependency of the dCB width on the value of the -boson mass in the  channel.It also shows a comparison of the model obtained with the prescription described above and the distribution of events obtained from an independent simulated sample.Similar tests show excellent modeling throughout the whole mass spectrum considered in this search.Once validated, a model built with all the simulated signal mass points was used in the analysis.

Systematic uncertainties
Several sources of experimental and modeling systematic uncertainty affecting the signal acceptance and efficiency and the background normalization are considered.The uncertainty in the combined 2015-2018 integrated luminosity is 1.7% [67], obtained using the LUCID-2 detector [68] for the primary luminosity measurements, complemented by measurements using the inner detector and calorimeters.The uncertainty in the number of pileup events is determined by comparing estimates based on the total inelastic cross section measured using either the luminosity detectors or the ID.
Uncertainties in the muon and electron efficiency, momentum scale and resolution are determined from alternative simulations, and from tag-and-probe measurements using  → ℓℓ and / → ℓℓ processes [57,60].The most important lepton uncertainties in this search are the ones related to the muon identification efficiency.These uncertainties are below 1% in the momentum range considered.
Uncertainties associated with the jet energy scale are evaluated by combining information from test-beam data, LHC collision data and simulation, and the jet energy resolution uncertainty is obtained from measurements of dĳet  T -balance in data and simulation [69].Additional considerations related to jet flavor, pileup corrections,  dependence and high- T jets are included.The efficiency to identify and remove jets from pileup is measured with  (→  +  − ) + jets events in data using techniques similar to those used in Ref. [62].The uncertainty in tagging -jets is 2%-10% depending on the jet  T .The uncertainty in mistagging -jets (light jets) is 10%-25% (15%-50%) depending on the jet  T [63][64][65].
Uncertainties in modeling the background are assessed primarily through variations of the renormalization and factorization scales.The effects of independent variations of the two scales between twice and half their nominal values are considered and their envelope is taken as representative of uncertainties associated with missing higher-order terms in the calculation of the hard-scatter ME.For the leading  t background process, the scale uncertainties are approximately 5%.Uncertainties associated with the showering and hadronization modeling are assessed through alternative Pythia 8 A14 tunes.Uncertainties in the electroweak production of a single top quark in association with a  boson are estimated by comparing different prescriptions to handle the interference with the  t process, and are found to have a negligible impact in this search.
Uncertainties associated with the signal processes are found to be largely independent of the -boson mass hypothesis, and a single uncertainty model is used independently of the signal hypothesis being tested.Uncertainties in modeling the signal acceptance are assessed by varying the renormalization and factorization scales, as described above, as well as through comparison of simulations using either Pythia 8 or Herwig 7.1 [70,71] for showering and hadronization.These uncertainties are approximately 7% for the cross section and 1% for the acceptance.Uncertainties stemming from variations of the proton PDFs are considered for both the signal and background simulations following the NNPDF prescription [28], and are found to have negligible impact on the result.
The systematic uncertainties are much smaller than the data statistical uncertainty.The two largest systematic uncertainties are those related to the muon identification efficiency and the modeling of the  t background.These uncertainties would have an impact of 2% and 1%, respectively, in the measurement of a hypothetical signal cross section, compared to a statistical uncertainty of approximately 40%-70%, depending on the -boson mass.

Results
The presence of a signal consistent with the production of  t or  →  + ,  + →  + , with  →  is tested by comparing the expected background with data in narrow bins of the reconstructed    .A total of 43 bins between 12 GeV and 77 GeV are used.For simplicity, the same binning is used in both the  and  regions.A bin width proportional to the expected width of a detected signal, illustrated in Figure 4(b), ensures that any signal contribution is highly concentrated in a few bins, while keeping the analysis largely independent of the specific signal mass distribution.
A test statistic is built from a profile likelihood ratio q = −2 ln(L (, θ )/L ( μ, θ)), where the likelihood L is built starting from the product of Poisson distributions in each analysis channel.The values of the parameters that maximize the likelihood function are μ and θ, and the values of the nuisance parameters that maximize the likelihood function for a given value of  are θ [72].The test statistic value is set to zero if the hypothesis  is lower than the maximum-likelihood estimator μ.The expected number of events is parameterized as a function of the signal strength , which multiplies a reference signal cross section of 1 fb, and a large number of nuisance parameters  describing the background model and systematic uncertainties.The parameters describing the background model include the ones describing the overall  t normalization, the background yield from processes with nonprompt leptons in three isolation sidebands independently for each analysis channel, and the three global fake factors.The analysis channels include the 43 bins in each signal region, but also the regions with various numbers of jets and -jets in the on- control region, and the three  ,fake T regions of the  t control region.Hypothesis testing is performed separately for each different value of the -boson mass.Nuisance parameters describing systematic uncertainties are included by following the prescription in Ref. [73] and have Gaussian constraint terms.Finally, a dedicated set of nuisance parameters is introduced to describe Poisson fluctuations in the background yields estimated from simulation.As shown in Figure 3, good agreement is observed between data and the expected background, suggesting the absence of a signal.A hypothetical  t signal with a cross section of 4 fb at   = 35 GeV is also depicted in Figure 3, showing the typical concentrated excess that would have been expected from a signal.No significant excess is observed, and the smallest local -value is 0.008 at   = 27 GeV, corresponding to a local significance of about 2.4 .The slight excess at   = 27 GeV is observed in both the  and  channels.Upper limits on the signal cross section are determined using the CL s prescription [72,74] and are shown in Figure 5(a) together with a comparison with a prediction from a theoretical model for the  t analysis.The model assumes a single coupling −    (t 5 )/ √ 2, where   is the top-quark Yukawa coupling, and the cross section is calculated at NLO precision in QCD.Only  →  decays are included.A value of   = 0.07 is chosen as an example in Figure 5(a) since it approximately reproduces the largest cross section not excluded by this analysis.The uncertainty bands include both the scale and PDF variations.In 2HDM+ models [54], if the heavy Higgs bosons decouple, the coupling   is given by sin /tan , where  is the mixing angle between the light and heavy pseudoscalar bosons, and tan  is the ratio of the vacuum expectation values of the two scalar doublets.

Conclusion
A search for a new pseudoscalar -boson produced in events with a top-quark pair where the -boson decays into a pair of muons,  → , is performed.The analysis is based on the dataset of   collisions at √  = 13 TeV recorded by the ATLAS detector in 2015 to 2018, corresponding to an integrated luminosity of 139 fb −1 .The search targets the final state where one top quark decays leptonically, resulting in two signal regions with three leptons, either  or , and multiple jets.No significant excess of events above the Standard Model expectation is observed and interpretations in two signal models are considered.Upper limits are set on the cross section for associated production of  t and a new pseudoscalar,   →  t, times the branching ratio B( → ) in the mass range 15 GeV <   < 72 GeV, and exclude signals with ( t)B( → ) above 0.5−3 fb at 95% confidence level.The search also sets upper limits on the branching ratio B( →  + ,  + →  + ,  → ) in the range (0.9−3.9) × 10 −6 at 95% confidence level for   + between 120 GeV and 160 GeV and   between 15 GeV and 72 GeV.The results are largely dominated by statistical uncertainties and are expected to improve as more LHC collision data are collected.
[59] ATLAS Collaboration, Reconstruction of primary vertices at the ATLAS experiment in Run  [64] ATLAS Collaboration, Measurement of -tagging efficiency of -jets in  t events using a likelihood approach with the ATLAS detector, ATLAS-CONF-2018-001, 2018, url: https://cds.cern.ch/record/2306649.

Figure 1 :
Figure 1: Feynman diagrams for the leading contributions to the (a)  t process and to the (b)   →  t process with  →  + ,  + →  + .

Figure 2 :
Figure 2: Comparison between data and expected background for the on- control region in the (a)  and (b)  final states.The bins correspond to different jet and -jet multiplicities.Rare background processes include  + jets,  ,   ,   , and  t t.The yields correspond to the values obtained under the background-only hypothesis with the profile likelihood method described in Section 9.

Figure 3 Figure 3 :
Figure3shows the background composition of the two signal regions.Background processes with one nonprompt muon are called -fakes and processes with one nonprompt electrons are called -fakes.Rare background processes with prompt leptons are shown together.

Figure 4 :
Figure 4: (a) Fit of the dCB width as a function of the -boson mass in the  channel.(b) Example fit of the signal using the dCB fit model compared with an independent simulated sample for the  t signal with   = 20 GeV in the  channel.

Figure 5 :
Figure 5: (a) Expected and observed 95% confidence level (CL) upper limits on the signal cross section shown as a function of   for  t production, and a comparison with the cross section predicted by the model described in the text.In (b), (c) and (d), expected and observed 95% CL upper limits on the branching ratio B( →  + ,  + →  + ,  → ) are shown as a function of   for  + →  + , assuming a top-pair cross section of (   →  t) = 833 pb [43-49], for three values of the charged Higgs boson's mass:   + = 120 GeV,   + = 140 GeV, and   + = 160 GeV, respectively.

Table 1 :
Generator setups for samples used in this analysis.The precision of the ME generator is NLO in QCD if no additional information is provided in parentheses.The higher-order cross section used to normalize these samples is listed in the last column if different from the one in the generator.

Table 2 :
Definition of the analysis regions, split according to the number of leptons, jets, and -jets, and the invariant mass requirements on muon pairs.