Collider Constraints on Lepton Flavor Violation in the 2HDM

In light of the recent CMS analysis on lepton flavor violating (LFV) heavy Higgs searches and updated bounds on various search channels involving neutral and charged scalars, we provide the updated constraints on the Type-III Two-Higgs-Doublet-Model (2HDM) with a $\tau-\mu$ LFV. In doing so, we first extend the CMS analysis to cover the mass region below 200 GeV by recasting their data. After obtaining the bounds on the heavy Higgs production in the mass range between 130 GeV and 450 GeV, we analyze the parameter space of the Type-III 2HDM with various scenarios of mass spectrum and heavy Higgs production strengths. We found that in most scenarios, searching for the heavy Higgs in the mass range lower than $2 m_W$ is very important in constraining the parameter space of the Type-III 2HDM. Hence, we suggest for the future analysis that the search window for the LFV heavy Higgs be extended to the lower mass region.


I. INTRODUCTION
The discovery of the 125-GeV resonance (h) resembling the Standard Model (SM) Higgs is arguably one of the most important findings in particle physics. Through a series of measurements, it has been demonstrated that the way the scalar interact with gauge bosons (i.e., γ, W , and Z) and third generation fermions is consistent with the SM expectations. However, one should note that the present data cannot rule out the presence of a new mechanism responsible for electroweak symmetry breaking, including the existence of additional scalars that may take place in the process. The effects of new physics can modify the 125-GeV Higgs couplings or so much as induce interactions that are completely absent in the SM.
The example of the latter is Higgs lepton-flavorviolating (LFV) couplings, responsible for decays like h → eµ or h → µτ . This type of processes can also correlate with the low-energy processes, like i → j γ. In fact, channels with tau lepton in final states, e.g., h → eτ and h → µτ , are found to be stronger in constraining the LFV couplings compared to present low-energy τ → e and τ → µ conversions, respectively [1]. Moreover, this particular channel has attracted great interests [2][3][4][5] and remains important for constraining many models beyond the SM. The most recent bound of this τ − µ channel, found by using 35.9 fb −1 data at 13 TeV center-of-mass energy, is set by CMS, i.e., BR(h → µτ ) < 0.25% at 95% CL [6]. A slightly weaker bound is obtained by ATLAS, BR(h → µτ ) < 0.28% [7].
One of the new physics model that can accommodate LFV is the Type-III Two-Higgs-Doublet-Model (2HDM).
In this model, the scalar content of the SM is enlarged by an additional electroweak doublet. As a result, the Yukawa coupling matrices of each fermion type can no longer be simultaneously diagonalized, so flavor-violating interactions naturally arise at tree level. 1 Furthermore, in addition to the h, there are more scalars, i.e., the heavy CP-even Higgs H, the pseudoscalar A, and the charged Higgs H + . Each scalar exhibits nontrivial flavor violation at tree level too. Previous work [5] (see also [4]) has shown that in the context of the current model, the heavy Higgs can have a much larger LFV branching fraction. Hence if the Higgs can be produced in a significant amount, this channel might provide a better opportunity to search for the LFV interactions.
Some of the LHC collaborations have recently started to search for LFV decays in this direction. The first LHC search for LFV decays of the new neutral Higgs was performed by the LHCb using 2 fb −1 of data at 8 TeV centerof-mass energy [9]. The LHCb search places a bound on the production cross-section times the τ -µ branching ratio (σ×BR(τ µ)) of the neutral Higgs between 4 pb to 22 pb over the mass range between 45 GeV and 195 GeV. This search is followed by a recent CMS analysis using 35.9 fb −1 of data at 13 TeV center-of-mass energy [10]. The CMS search covers a mass range between 200 GeV and 900 GeV. Their reported bounds are between 51.9 fb to 1.6 fb. Even though both analyses cover different mass regions, we expect the CMS search to provide a stronger bound than the LHCb in the case the scalar mass is lighter than 200 GeV. This is due to the fact that the CMS analysis employs more data and has higher acceptance.
This work consists of two parts based on the recent CMS results. First, we show that the CMS results can be extended to a lower mass region. We recast the CMS results to cover the scalar mass between 130 GeV and 450 GeV. Then, we study the implication of recast CMS bounds on the allowed parameter space of the Type-III 2HDM. In particular, we will compare the heavy scalar LFV bounds against the h LFV bounds and the bounds for other non-LFV searches. By comparing the excluded parameter space from all the searches, we will show the importance of the LFV heavy Higgs searches in various regions of the Type-III 2HDM parameter space.

II. RECASTING THE CMS ANALYSIS
In this section we recast the CMS results to cover the mass region lower than 200 GeV. We will see that the lower mass bounds is important for excluding the parameter space of the Type-III 2HDM, especially in the case of heavy Higgs produced with a small cross-section.  [10].
In order to simulate the signal, we use MadGraph 5 [11] followed by shower and hadronization simulation by Pythia 8 [12].
Detector simulation is done using Delphes [13] and we use MadAnalysis 5 [14] to analyze the simulated events. The cuts used in the CMS analysis are shown in Table I. The comparison between our montecarlo simulation and CMS simulation for m H = 200, 300 and 450 GeV is shown in Fig. 1. In the plot, the collinear mass, M col , is defined as The parameter x vis is defined as x vis = | p τvis T | / | p τvis T | + E miss T ·p τvis T , where p τvis T is the transverse momentum of the visible decay of the tau (τ h or e).
In order to derive the bound, we compare the simulated signal and the background. We follow the strategy of [15] in calculating the likelihood. The likelihood of observing the signal in a particular bin is given by where n i is the number of observed event in the bin, s i is the predicted number of signal events and b i is the predicted number of background events. The probability P i (ξ) is given by the log-normal function where σ i is the relative systematic uncertainties of the corresponding bin. The chi squared value is given by The 95% C.L. bounds on the heavy Higgs σ×BR(τ µ) for m H between 130 GeV to 450 GeV are shown in Fig. 2. The cuts utilized by CMS still allow a reasonable acceptance for m H < 200 GeV. Hence from our recast, we show that the CMS data can exclude the region of m H between 130 GeV to 200 GeV better than the LHCb official analysis, which uses less data. Note that our recast analysis gives slightly worse bounds compared to the official CMS analysis for m H ≥ 200 GeV, due to the simplistic nature of the statistics we employ. In Sec. IV we will examine the bounds from the combination of scalar and pseudoscalar productions in our benchmark scenarios.

III. TYPE-III TWO-HIGGS-DOUBLET-MODEL
In this section, we give a brief overview of the Type-III 2HDM. We will closely follow the convention of Ref. [5], to which the readers may refer to for a detailed description of the model.

A. The Model
The hypercharge of both scalar doublets Φ 1 and Φ 2 are taken to be 1/2. For convenience in discussing flavor violation, we employ the Higgs basis [16] in which the electroweak vacuum expectation value (vev) resides only in Φ 1 . In unitary gauge, the doublet Φ 1 and Φ 2 can be The LHC bounds from LHCb [9], CMS [10] and our recast of CMS analysis [10].
expanded as where v = 246 GeV is the electroweak vev. The CPeven φ 1 and φ 2 can be rotated to the mass basis by an orthogonal rotation where c α (s α ) is a shorthand for cos α(sin α).
The Yukawa couplings of Φ 1 are responsible for fermion mass generation while the Yukawa couplings of Φ 2 induce potential flavor violations. Writing them explicitly, we have [5] where m f are the diagonal fermion mass matrices, Y f are the Yukawa coupling matrices,Φ = iσ 2 Φ * , V is the Cabibbo-Kobayashi-Maskawa matrix and we have suppressed the flavor indices. Note in the above equation all the fermions are in the mass eigenstates with In the rest of this paper, for brevity, we will refer to the h as the light Higgs and the H as the heavy Higgs. We will also refer to H, A and H + collectively as the heavy scalars.

B. Productions and Decays of Heavy Scalars
The light Higgs couplings to SM particles have been measured extensively at the LHC [17]. They are consistent with the SM expectations. Thus one would expect the mixing angle α to be small, see Eq. (5). The small mixing angle has a significant impact on the production of the heavy scalars as we will now discuss. The production cross-section of the H and A are given in Ref. [5]. The production channels of the H are similar to those of the light Higgs. On the other hand, the A is mainly produced by gluon fusion and associated production with a pair of top quarks while the H + is produced in association with a top quark and a bottom quark. Here we note that the Φ 2 Yukawa couplings, especially Y tt U , allow for sizeable production cross-sections of the H, the A and the H + . 2 In the absence of such couplings, a single-scalar productions of A and H + vanish. However, the single-scalar production of H can still proceed via the mixing in a neutral component of Φ 1 . Nevertheless, this production cross-section is suppressed by s 2 α . Thus we will loosely refer to the scenario where Y tt U is present as a large production cross-section case. Similarly, we will loosely refer to the case where Y tt U = 0 as the small production cross-section scenario. The production crosssections of the heavy scalars for both cases are shown in Fig. 3.
The partial decay widths of the heavy neutral scalars are also given in Ref. [5]. We note here that in the absence of the Φ 2 Yukawa couplings (the Y 's), the H would 2 The presence of any of the Y U 's or Y D 's would enhance the H production. However, the light quark Yukawas are strongly constrained by dijet searches [18] and the 125 GeV Higgs data [19]. Moreover, light quark Yukawas would dilute the H LFV branching ratios. Therefore, we choose to consider only the Y tt U coupling.
The LFV decays of the heavy scalars are mediated by the Y couplings. In this work, we will focus on flavor violation in the τ -µ sector which arise from the Y µτ and Y τ µ couplings. For simplicity, we will take Y µτ = Y τ µ to be real. If other components of Y , such as Y τ τ , are non-vanishing, they will dilute the τ -µ flavor violating branching ratios of the heavy scalars. The branching ratios of the heavy scalars are shown in Fig. 4.

C. LHC Searches for Heavy Scalars
The heavy scalars have been extensively searched for at the LHC. In particular CMS has been searching for the s-channel production of the heavy Higgs decaying into a tau-pair [20] and W -pair [21]. The former analysis covers the mass region m H > 90 GeV while the latter covers m H > 200 GeV. Moreover, CMS has performed a search for a pseudoscalar decaying into a pair of hZ [22]. All three analyses are based on 35.9 fb −1 of the LHC Run II data.
The charged Higgs can be searched for in the decay channel H + → tb in the case of a heavy charged Higgs, m H + > m t + m b , and H + → τ ν in the case of a light charged Higgs. The former search has been performed by ATLAS [23] using 36.1 fb −1 of the 13 TeV LHC data while the latter search has been analyzed by CMS [24] using 35.9 fb −1 of data.
We will collectively refer to the constraints placed by the above searches on the parameter space of the Type-III 2HDM as the non-LFV bounds. In the next section, we will only show the strongest constraint among all the analyses above.

IV. LHC CONSTRAINTS ON τ -µ LFV
In this section, we will present the viable parameter space of the Type-III 2HDM with LFV in the τ -µ sector with respect to collider searches. We start by computing the σ×BR(τ µ) of the heavy Higgs as a function of its mass and comparing it against the recast CMS analysis. In Fig. 5, the σ×BR(τ µ) is shown for both the small production case (Y tt U = 0) and the large production case (Y tt U = 0.5). In making these plots we take sin α = 0.1 and λ Hhh = 0.5. From the plot, we see that in the small production case, the CMS bounds become irrelevant once the phase space for H → W + W − opens, i.e. m H 160 GeV. In the strong production case, the bounds can be extended beyond the W + W − threshold. In our particular benchmark case, the bound extends to around m H ∼ 2m h , where the decay H → hh opens. However the bound could be extended further if the coupling λ Hhh is smaller, which in turn enhances the τ -µ branching ratio of the heavy Higgs.
In the rest of this section, we will further analyze the implication of the CMS heavy Higgs τ -µ LFV search on the Type-III 2HDM. We will also study the interplay between the light Higgs LFV search, the heavy Higgs LFV search and the heavy scalar searches in constraining the model parameter space.

A. Large Production Case
In this subsection, we analyze the most optimistic scenario in which the heavy scalars production is enhanced, while their branching fractions to τ -µ are not diluted by other decay channels. This scenario corresponds to keeping only Y tt U and Y τ µ = Y µτ . The viable parameter space of the model depends on the scalar mass spectrum. To simplify the discussion, we consider three representative cases: 1) the H is the lightest among the heavy scalars, 2) the A is the lightest amongst the heavy scalars, and 3) all heavy scalars are degenerated. For the first two cases, we will take the remaining two heavy scalars to be degenerated in mass and are 100 GeV heavier than the lightest one. In all the above representative cases, we have verified that the scalar spectrum is consistent with the electroweak S and T parameters over the mass range considered in our analyses.

Lightest H
We will first discuss the case where the scalar H is the lightest with m H + = m A = m H + 100 GeV. The exclusion regions in the Y µτ -Y tt U plane for sin α = 0.05 and 0.1, together with m H = 130, 200 and 300 GeV, are shown in Fig. 6. Comparing the left and the right plots, we see that the h LFV bounds become less relevant as sin α gets smaller.
We also note that, for a fixed sin α, the bounds from the heavy Higgs LFV search get stronger as m H becomes lighter. In the case of m H = 130 GeV, the bounds from the heavy Higgs LFV search exclude most of the parameter space, except when either Y µτ becomes very small or the combination of sin α and Y tt U conspires to have the H production cross-section vanished. Only in the case of a very heavy H, the constraints from the LFV decay of the light Higgs become relevant in most of the parameter space.
The non-LFV constraints depends heavily on the mass. For m H = 130 GeV (m H + = m A = 230 GeV), the most relevant constraint is the charged Higgs search in the H + → tb channel. For m H = 200 GeV (m H + = m A = 300 GeV), the search for pseudoscalar decaying into hZ provides a more stringent constraint on the parameter space compared to the H + → tb search. Finally, for m H = 300 GeV (m H + = m A = 400 GeV), the pseudoscalar decays mostly into tt, diluting the A → hZ signal. As a result, the H + → tb search is again becoming the most constraining search.

Lightest A
We turn next to the case in which the pseudoscalar A is the lightest with m H + = m H = m A + 100 GeV. The exclusion plots for this case are shown in Fig. 7. We see that the pseudoscalar LFV searches generically provide better constraints compared to the h LFV searches. Several comments on the bounds are in order. For the pseudoscalar mass below the hZ threshold, the A decays mostly into τ µ. Thus the pseudoscalar LFV bounds for m A = 130 and 200 GeV are stronger than the corresponding H LFV bound with the same mass. For m A = 300 GeV, the A → hZ is open, diluting the A → τ µ signal. However, for our scenario, the branching ratio A → τ µ with m A = 300 GeV is still larger than the corresponding branching ratio for the H with the same mass. As a result, the pseudoscalar LFV search provides a better constraint in this case as well. Lastly, we note that the production cross-section of the the pseudoscalar does not depend on the mixing angle sin α. Therefore the heavy Higgs LFV search bounds, shown in the red lines on the left and the right plots of Fig. 7, do not change significantly as we vary the value of the mixing angle.
We conclude this subsection by discussing the non-LFV bounds. For m A ≤ 200 GeV, the most relevant constraint comes from the charged Higgs search in the tb channel. For m A = 300 GeV, the decay channel A → hZ opens. Hence the search for A → hZ provides the most stringent non-LFV constraint in this case.

Degenerate Spectrum
In this subsection we consider the case where all the heavy scalars are degenerated. The viable parameter space in this case is shown in Fig. 8. We see that the bounds from the heavy Higgs LFV search become stronger compared to the two previous benchmarks because now the two heavy neutral scalars are equally light and can be produced with larger cross-section. Similarly the bounds from the non-LFV heavy scalar searches also become stronger. On the other hand, the bounds from the light Higgs LFV search are independent of the heavy scalar spectrum, hence they do not change from the previous benchmarks. We note that in the degenerate case, the heavy scalar LFV searches generically provide more significant constraints compared to the light Higgs LFV search. For our particular benchmark, the bounds are mostly driven by the pseudoscalar searches since its LFV branching ratio is bigger than that of the heavy Higgs. It is worth noting that for sufficiently light heavy scalars, such as m H = m A = 130 GeV, with a large enough mixing, the heavy scalars LFV search places a very strong bound on the Y τ µ and Y µτ . For m H = m A = 130 GeV with sin α = 0.1, Y τ µ and Y µτ are constrained to be less than 0.003, see the right plot in Fig. 8.
Let us end this subsection with a few remarks regarding the non-LFV searches. The most relevant constraint  [6]. The allowed regions from the CMS heavy Higgs LFV search [10] are denoted by the area between the two red lines for each corresponding mH . The allowed regions from the non LFV heavy scalar searches are denoted by the area between the green lines. The dash-dotted, dashed and solid lines represent the boundaries of the excluded regions for mH = 130, 200 and 300 GeV respectively.   [6]. The allowed regions from the CMS heavy Higgs LFV search [10] are denoted by the area between the two red lines for each corresponding mH . The allowed regions from the non LFV heavy scalar searches are denoted by the area between the green lines. The dash-dotted, dashed and solid lines represent the boundaries of the excluded regions for mH = 130, 200 and 300 GeV respectively. in this case depends strongly the heavy scalar mass. For the case m H = 130 GeV, the most constraining bound comes from the charged Higgs search in the τ ν channel. For m H = 200 GeV, the charged Higgs search in the tb channel provides the strongest constraint. Finally, in the case m H = 300 GeV, the phase space for A → hZ opens. Hence the A → hZ search provides the strongest bounds.

B. Small Production Case
In the previous section, we have discussed the case where the production of the heavy scalar is enhanced by the presence of Y tt U . In that optimistic scenario, we see that the CMS heavy Higgs LFV search can put bounds  [6]. The allowed regions from the CMS heavy Higgs LFV search [10] are denoted by the area between the two red lines for each corresponding mH . The allowed regions from the non LFV heavy scalar searches are denoted by the area between the green lines. The dash-dotted, dashed and solid lines represent the boundaries of the excluded regions for mH = 130, 200 and 300 GeV respectively. on a large section of the parameter space unconstrained by the light Higgs LFV search. In this section, we discuss the relevance of the heavy Higgs LFV search in the pessimistic scenario where Y tt U is absent. In this scenario, only the heavy Higgs can be produced via the neutral CP-even scalars mixing. We introduce two benchmarks, the first is the less pessimistic case where the H → τ µ branching fraction does not get diluted. We will refer to this benchmark as the mixing production case. The second benchmark represents the most pessimistic case where the H → τ µ branching fraction is diluted by the presence of Y τ τ . We will refer to this benchmark as the mixing production with Y τ τ case.

Mixing Production
In this subsection, we consider the case in which all Y 's except Y τ µ and Y µτ are vanishing. The scalar H can only be produced via the mixing in the neutral component of doublet Φ 1 . Its production cross-section is controlled only by the mixing angle sin α and the mass m H . As for the pseudoscalar, since Y tt U is zero, it cannot be produced via gluon fusion and hence does not contribute to the constraints on the parameter space. Moreover, non-LFV heavy Higgs searches discussed in Sec. III C require that Y tt U = 0. Therefore, they also do not provide any constraints in this scenario.
The excluded parameter space is shown in Fig. 9. The bounds are mostly insensitive to the heavy scalar mass spectrum, as long as the H does not decay into other scalars. From the plot we can see that only in the case of m H 2m W , the heavy Higgs LFV search becomes relevant. For m H 2m W , the heavy Higgs decays dominantly into W W , diluting the LFV signal and weakening its bound. Hence the heavy Higgs LFV search is only relevant for the case of m H 2m W , which was not considered in the official CMS analysis [10].  [6]. The red region represents the boundary of excluded regions from the CMS heavy Higgs LFV search [10]. All the area on the right of the respective red lines are excluded. The light blue region is excluded by the signal strength measurement of light Higgs decay to τ + τ − .

Mixing Production With Y τ τ
In this subsection, we suppress the H → τ µ branching fraction by introducing an extra lepton Yukawa Y τ τ . There are several constraints when the Y τ τ is present. The Y τ τ is constrained by the signal strength measurement of h → τ + τ − . The latest measurements by both ATLAS and CMS give µ = 0.82 ± 0.16 [17]. Moreover, the presence of Y τ τ will induce a τ → µγ decay. However, we find that the constraint from τ → µγ decay is negligible compared to the LFV collider bounds for our benchmark scenarios. Fig. 10 shows the bounds on the model parameter space in the Y µτ and Y τ τ plane. The plot shows that even in this case, the heavy Higgs LFV search can still provide substantial bounds, provided that the value of sin α is large enough and the heavy Higgs mass is below the W W threshold. In this case, the H → τ µ branching fraction is only suppressed by the decay into τ + τ − which, while significant, is still on the same order as the H → τ µ decay.

V. CONCLUSIONS
In this work we have studied the significance of the recent CMS heavy Higgs LFV search [10]. First, we recast the CMS analysis to obtain the σ×BR(τ µ) bounds for the heavy Higgs mass between 130 GeV and 450 GeV. Then we study the implication of the bounds in the context of the Type-III 2HDM. We consider the most optimistic and pessimistic scenarios for probing the τ -µ LFV. In the optimistic scenarios, the production cross-sections of the heavy scalars are large because of the presence of Y tt U . In this case, we have shown that the official CMS heavy Higgs LFV search generically extends the exclusion regions of the model parameter space of the light Higgs LFV search. Moreover, the bounds from the heavy Higgs LFV search are especially strong when the heavy scalars are light. In the pessimistic scenarios, in which the heavy Higgs is produced by the mixing with the light Higgs, we found that the heavy Higgs LFV search still plays an important role in probing the parameter space of the model. However, in this case the heavy Higgs LFV search is only relevant for m H 2m W , i.e., below the W + W − decay threshold, see Fig. 9. Unfortunately, this low mass region does not get covered by the official CMS search which only considers m H > 200 GeV. Thus, we suggest CMS to consider extending their heavy Higgs mass region to below the W + W − threshold in their future analysis.