Flavon Signatures at the HL-LHC

The detection of a single Higgs boson at the Large Hadron Collider (LHC) has allowed one to probe some properties of it, including the Yukawa and gauge couplings. However, in order to probe the Higgs potential, one has to rely on new production mechanisms, such as Higgs pair production. In this paper, we show that such a channel is also sensitive to the production and decay of a so-called `Flavon' field ($H_F$), a new scalar state that arises in models that attempt to explain the hierarchy of the Standard Model (SM) fermion masses. Our analysis also focuses on the other decay channels involving the Flavon particle, specifically the decay of the Flavon to a pair of $Z$ bosons ($H_F \to Z Z$) and the concurrent production of a top quark and charm quark ($H_F\to tc$), having one or more leptons in the final states. In particular, we show that, with 3000 fb$^{-1}$ of accumulated data at 14 TeV (the Run 3 stage) of the LHC an heavy Flavon $H_F$ with mass $M_{H_F} \simeq 2m_t $ can be explored with $3\sigma -5\sigma $ significance through these channels.


I. INTRODUCTION
The discovery of a Higgs boson [1][2][3] with mass M h = 125.5 GeV has provided a firm evidence for the mechanism of Electro-Weak Symmetry Breaking (EWSB) based on a Higgs potential [4,5] pointing towards the minimal realization of it that defines the Standard Model (SM).So far, the corresponding studies have relied on the four standard single Higgs production mechanism, i.e., gluon-gluon fusion, vector boson fusion, Higgs-strahlung and associated production with topquark pairs (see Ref. [6]), which have permitted to extract the Higgs boson couplings with quarks (b and t), leptons (τ and µ) and gauge bosons (W and Z) as well as the effective interaction with photon and gluon pairs.However, there still remains the task of probing the Higgs selfcoupling.Moreover, we still do not understand the origin of the Yuwaka couplings, the flavor coupling.The Higgs boson pair (hh) production serves as a direct means of investigating the self-interactions of the Higgs boson, which play a crucial role in determining the Higgs potential of the SM.Additionally, the magnitude of the hh production rate is directly proportional to the square of this self-coupling.Within the SM, the non-resonant production of Higgs boson pairs represents the only direct method for measuring the Higgs boson self-coupling.Nonetheless, due to the limited size of the cross section, accurately determining this coupling presents a significant challenge.Next-to-Leading Order effects help somewhat to improve the situation [7][8][9].The production of SM-like Higgs boson pairs at the LHC provides a valuable avenue to probe various scenarios Beyond the SM (BSM) that contain particles having couplings with the Higgs boson [10][11][12].These new particles could be (pseudo)scalars, fermions and gauge bosons.Thus di-Higgs production offers insights into the properties of the Higgs boson itself and can potentially shed light on the Higgs self-interactions as well as its interactions with other particles in the model.For example, in the dominant production mode for di-Higgs bosons at the LHC, through fusion of gluons, mediated by top quark loops that couple to both gluons and Higgs boson.Any additional heavy coloured fermions that couple with the Higgs boson can contribute to the di-Higgs production mode too.Similarly, in some BSM scenarios, Higgs production can be associated with other coloured particles in the loops, such as squarks in Supersymmetry.
Studies have shown that, in all generality, in scenarios with an extended Higgs sector, new heavy resonances, supersymmetric theories, effective field theories with modified top Yukawa coupling, etc., di-Higgs (hh) and di-gauge boson(W + W − /ZZ) production receives additional BSM contributions along with the SM ones .These effects make the study of these two production processes particularly interesting then and, at the same time, also very challenging.However, the possibility to produce Higgs and gauge bosons pairs in the decay of a new heavy particle that belongs to the spectrum of those models offers some hope to achieve detectable signals at current and future colliders.It is also to be noted that flavor-violating Higgs decays, i.e., those violating the conservation of flavor quantum numbers [52] can be possible.This phenomenon is of great interest as it can provide evidence for BSM physics and shed light on the origin of flavor mixing and hierarchy in the fermion sector.The study of flavor-violating Higgs decays thus offers a unique opportunity to explore new physics and deepen our understanding of fundamental interactions in the universe [53][54][55].
Specifically, we will study the interactions of the discovered Higgs boson with the so-called 'Flavon' field H F which appears in models that attempt to explain the hierarchy of quark and lepton masses using the Froggatt-Nielsen (FN) mechanism [56][57][58].This mechanism assumes that, above some scale M F roughly corresponding to the Flavon mass, there is a symmetry, perhaps of Abelian type U (1) F , with the SM fermions being charged under it, which then forbids the appearance of Yukawa couplings at the renormalizable level.However, Yukawa matrices can arise through non-renormalizable operators.The Higgs spectrum of these models includes a light H F state, which could mix effectively with the SM Higgs boson when the flavor scale is of the order 1 TeV or lower.Recently, the phenomenology of Higgs vs Flavon interactions at particle colliders has been the focus of some attention [59][60][61][62][63][64][65][66].In particular, within this framework, it is possible to have a coupling of this new scalar with Higgs and gauge bosons pairs, which can then provide interesting signals to be searched for at the LHC.Another characteristic to highlight is the emergence of Flavor Changing Neutral Currents (FCNCs) mediated by the Flavon, which allows the H F → tc decay at tree level.Our study could thus not only serve as a strategy for the Flavon search, but it can also be helpful to assess the order of magnitude of flavor violation mediated by such a particle, which is an indisputable signature of BSM physics.
In this paper, we are interested in studying the detection of the Flavon signal emerging from the production and decay processes pp and the FCNC process pp → H F → tc (t → ν b) at future stages of the LHC, namely, Run 3 and the High-Luminosity LHC (HL-LHC) [67,68].In this analysis, we do not take into consideration the pp → H F → hh (h → τ + τ − , h → b b) channel, which has the potential to be competitive with our selected signal.We have opted to exclude this channel from the current study and instead reserve it for a future publication.Additionally, we will not be presenting other channels such as pp → H F → hh (h → b b) and pp → H F → W W (W → ν ) because these channels are highly suppressed by large SM backgrounds.
The ATLAS and CMS collaborations at the LHC have already performed several studies of non-resonant di-Higgs and di-boson(W/Z) production with various possible final states using both Run 1 and the Run 2 dataset.None of these searches have observed a statistically significant excess over the SM background, therefore, upper limits on the di-Higgs production cross section are placed [69][70][71][72][73][74][75][76][77][78][79][80][81].We focus here on the '2 γ plus 2 b-jets', '2 pairs of same flavor opposite sign (SFOS) leptons' and '2 jets plus a charged lepton with its neutrino' (with one of the jets labeled as a b-jet) signatures.These particular (and comparatively clean) final states are obtained through pp → H F → hh, pp → H F → ZZ and pp → H F → tc production followed by h → γγ, h → b b, Z → ¯ and t → ν b decays.We will show that these channels have large significances in specific parameter space regions in the context of the LHC operated at √ s = 14 TeV of energy with integrated luminosity 3000 fb −1 .Besides these future energies and luminosities, we also present our results based on the data set accumulated to date, i.e., with a luminosity of 139 fb −1 at the 13 TeV LHC (Run 2).
The advocated signature of SM di-Higgs (hh) and di-boson (ZZ) processes have been explored earlier in the literature, albeit in different scenarios , while the processes tackled here, pp → H F → hh (h → γγ, h → b b) and pp → H F → ZZ (Z → ¯ ) and pp → H F → tc in the context of the present model have not been discussed in any depth [59][60][61][62][63][64].Our analysis of these final states give promising results as a discovery channel for a heavy CP-even H F boson in the aforementioned FN framework.In order to prove this, we first choose three sets of reference points for three heavy Higgs masses 800, 900 and 1000 GeV.A signal region (a set of different kinematic cuts) is then defined to maximize signal significances in the presence the SM backgrounds having the same final state.In our cut-based analysis, we further use the same signal region for different combinations of the singlet scalar Vacuum Expectation Value (VEV) v s and heavy Higgs mass M H F to compute the signal significances.The latter are only mildly affected (at the 5 − 10% level) by incorporating a realistic 5% systematic uncertainty in the SM background estimation.We find a large number of signal events that have significances exceeding 2σ and they can be explored with 3000 fb −1 of data at LHC runs using √ s = 14 TeV.
The rest of the paper is organized as follows.In sec.II, we present the details of the model and derive expressions for the masses and relevant interaction couplings for all the particles.Afterwards, we introduce the constraints acting on it from both the theoretical and experimental side in sec.III.Sec.IV is focused on the analysis of the signals arising from the decay of the Flavon.Finally, we conclude in sec.V.

II. THE MODEL
We now focus on some relevant theoretical aspects of what we will refer to as the FN singlet Model (FNSM).In Ref. [82], a comprehensive theoretical analysis of the Higgs potential therein is presented along with the constraints on the parameter space from the Higgs boson signal strengths and the oblique parameters, including presenting a few benchmark scenarios amenable to phenomenological investigation.(See Ref. [83] for the effects of Lepton Flavor Violation (LFV).)

A. The scalar sector
The scalar sector of this model consists of the SM Higgs doublet Φ ane and one SM singlet complex FN scalar S F .In the unitary gauge, we parameterize these fields as: where v and v s represent the VEVs of the SM Higgs doublet and FN singlet, respectively.The scalar potential should be invariant under the FN U (1) F flavor symmetry.Under this symmetry, the SM Higgs doublet H and FN singlet S F transform as Φ → Φ and S F → e iθ S F , respectively.
In general, such a scalar potential admits a complex VEV, S F 0 = vs √ 2 e iξ , but in this work we consider the special case in which the Higgs potential is CP-conserving, by setting the phase ξ = 0.Such a CP-conserving Higgs potential is then given by: 3) The U (1) F flavor symmetry of this scalar potential is spontaneously broken by the VEVs of the spin-0 fields (Φ, S F ) and this leads to a massless Goldstone boson in the physical spectrum.In order to give a mass to it, we add the following soft U (1) F breaking term to the potential: The full scalar potential is thus: The presence of the λ 3 term allows mixing between the Flavon and the Higgs fields after both the U (1) F flavor and EW symmetry breaking and contributes to the mass parameters for both the Flavon and Higgs field, as can be seen below.The soft U (1) F flavor symmetry breaking term V soft is responsible for the pseudoscalar Flavon (S I ) mass.Once the minimization conditions for the potential V are applied, we obtain the following relations between the parameters of V : All the parameters of the scalar potential are real and therefore the real and imaginary parts of V do not mix.The CP-even mass matrix can be written in the (φ 0 , S R ) basis as: The corresponding mass eigenstates are obtained via the standard 2 × 2 rotation: ) with α a mixing angle.Here h is identified with the SM-like Higgs boson with mass M h =125.5 GeV whereas the mass eigenstate H F is the CP-even Flavon.The corresponding CP-odd Flavon A F ≡ S I will have a mass such that M 2 A F = 2m 2 3 .Both H F and A F are considered to be heavier than h.In this model, we will work with the mixing angle α and physical masses M h , M H F and M A F , which are related to the quartic couplings of the scalar potential in Eq. (2.3) as follows: ) We consider the mixing angle α, the FN singlet VEV v s and its (pseudo)scalar field masses M H F ,A F as free parameters in this work.

B. The Yukawa sector
The effective U (1) F invariant Yukawa Lagrangian, á la FN, is given by [58]: where ρ u/d/ are dimensionless couplings seemingly of order one.This will lead to Yukawa couplings once the U (1) F flavor symmetry is spontaneously broken.The integers q f ij (f = u, d, ) are the combination of U (1) F charges of the respective fermions.In order to generate the Yukawa couplings, one spontaneously breaks both the U (1) F and EW symmetries.In the unitary gauge one can make the following first order expansion of the neutral component of the heavy Flavon field S F around its VEV v s : which leads to the following fermion couplings after replacing the mass eigenstates in L Y : where we define sin α ≡ s α and cos α ≡ c α .Here, M f stands for the diagonal fermion mass matrix while the intensities of the Higgs-Flavon couplings are encapsulated in the Zf matrices.In the flavor basis, the Z f ij matrix elements are given by: which remains non-diagonal even after diagonalizing the mass matrices, thereby giving rise to FV scalar couplings.In addition to the Yukawa couplings we also need the φV V (V = W, Z) couplings for our calculation which can be extracted from the kinetic terms of the Higgs doublet and complex singlet.In Tab.I we show the coupling constants for the interactions of the SM-like Higgs boson and the Flavon to fermions and gauge bosons.

III. CONSTRAINTS ON THE FNSM PARAMETER SPACE
In order to perform a realistic numerical analysis of the signals analyzed in this work, i.e., pp 1), we need to constrain the free FNSM parameters, i.e.: (i) the mixing angle α of the real components of the doublet Φ and the FN singlet S, (ii) FN singlet VEV v s , (iii) the heavy scalar(pseudo) field masses M H F ,A F , (iv) the diagonal Zu 33 ≡ Ztt , Zu 22 ≡ Zbb and the nondiagonal Zu 32 ≡ Ztc matrix elements which will be used to evaluate both the production cross section of the Flavon H F and the decay of the Higgs boson to a pair of b quarks; all of which have an impact on the upcoming calculations.These parameters are constrained by various kinds of theoretical bounds like absolute vacuum stability, triviality, perturbativity and unitarity of scattering matrices and different experimental data, chiefly, LHC Higgs boson coupling modifiers, null results for additional Higgs states plus the muon and electron anomalous magnetic (dipole) moments ∆a µ and ∆a e , respectively.The various LFV processes τ → 3µ, µ → 3e, τ → µγ, µ → eγ, B 0 s → µ + µ − and the total decay width of the Higgs boson (Γ h T ) are also modified in the presence of these new Yukawa couplings, so they have also been tested against available data.In the following, we discuss the various constraints on the model parameters in turn.

A. Stability of the scalar potential
The absolute stability of the scalar potential in Eq. ( 2.3) requires that the potential should not become unbounded from below, i.e., it should not approach negative infinity along any direction of the field space (h, H F , A F ) at large field values.Since in this limit the quadratic terms in the scalar potential are negligibly small as compared to the quartic terms, the absolute stability conditions are [84]: wherein these quartic couplings are evaluated at a scale Λ using Renormalization Group Evolution (RGE) equations.If the the scalar potential in Eq. ( 2.3) has a metastable EW vacuum, then these conditions are modified [84].One can then use Eq.(2.11) to translate these limits into those on the free parameters such as scalar fields' mass and mixing angles.

FIG. 2:
In the first two plots we show the perturbative bounds on the quartic couplings λ 2,3 while the third plot shows the stringent unitary bounds on λ U .

B. Perturbativity and unitarity constraints
To ensure that the radiatively improved scalar potential of the FNSM remains perturbative at any given energy scale, one must impose the following upper bounds on the quartic couplings: The quartic couplings in the scalar potential of our scenario are also severely constrained by the unitarity of the Scattering matrix (S-matrix).At very large field values, one can get the S-matrix by using various (pseudo)scalar-(pseudo)scalar, gauge boson-gauge boson and (pseudo)scalargauge boson interactions in 2 → 2 body processes.The unitarity of the S-matrix demands that the eigenvalues of it should be less than 8π [84,85].In the FNSM, the unitary bounds are obtained from the S-matrix (using the equivalence theorem) as: We now use the relation in Eq. (2.11) to display theoretical bounds on the scalar singlet VEV v s for various values of the heavy Higgs masses, M H F and M A F .In Fig. 2 we display the constraints on scalar quartic couplings coming from the perturbativity (Fig. 2(left) & (middle)) and unitarity (Fig. 2(right)) of the S-matrix.Here, we assume M H F = M A F and cos α = 0.995, which agrees with the constraints from the Higgs boson coupling modifiers from the LHC measurements, which we will discuss in some detail later.Fig. 2(left) shows the v s − λ 2 plane for M H F = 200, 400, 600, 800 and 1000 GeV whereas in Fig. 2 2(right) shows the unitary bounds.We find that |λ U | ≤ 16π/3 is the most stringent upper bound for the scalar quartic couplings.From these plots, we can see that the lower limit on the scalar singlet VEV v s is, for M H F = (200, 400, 600, 800, 1000) GeV, v s ≥ (69,138,207,276,345) GeV.Note that we are working at the EW scale only, as detailed RGE analysis is beyond the scope of this work.We also choose the parameters in such a way that the scalar potential remains absolutely stable in all the directions of the scalar fields h, H H , A F .(Further details can be found in Ref. [84].)To constrain the mixing angle α and the VEV of the FN singlet v s , we use HL-LHC projections for the Higgs boson coupling modifiers κ i at a CL of 2σ [86], as this machine configuration is the one with highest sensitivity among those we will consider in the analysis section.For a production cross section σ(pp → φ) or a decay width φ → X (φ = h, h SM ), we introduce: where X = b b, τ − τ + , W − W + , ZZ, γγ.Fig. 3(a) shows all the regions complying with the aforementioned projections for each channel in the cos α − v s plane: here, the green, pink, blue, orange and cyan area corresponds to κ b , κ τ , κ V , κ γ and κ g , respectively, while the red area represents the intersection of all the areas allowed by all the individual channels.We consider Zbb = 0.01 and Ztt = 0.4 in the evaluations for the κ X .Such values are well motivated because they simultaneously accommodate all the κ X 's.In fact, values in the 0.01 ≤ Zbb ≤ 0.1 and 0.1 ≤ Ztt ≤ 1 intervals have no important impact on the coupling modifiers, however, in the case when Zbb ≥ 0.1 and Ztt ≥ 2, a large reduction of allowed values in the cos α − v s plane is found [64,65].Furthermore, we present in Fig. 3(b) the cos α−v s plane regions allowed by ∆a µ (black points), ∆a e (magenta points), µ → 3e (red points) and B 0 s → µ + µ − (blue area).We have also analyzed the decays τ → 3µ, τ → µγ, µ → eγ, however, these processes are not very restrictive in the FNSM.This is mainly due to the choice we made for the matrix elements Zµµ and Zττ , as they play a subtle role in the couplings (see Tab. I) φµ − µ + and φτ − τ + (φ = h, H F , A F ), which have a significant impact on the observables τ → 3µ, τ → µγ, µ → eγ.In fact, we use Zττ = 0.2 and Zµµ = 10 −4 (hence, a strong hierarchy), otherwise the SM hµ − µ + coupling would be swamped by new corrections due to the FNSM 1 .So the bounds coming from the processes τ → 3µ, τ → µγ, µ → eγ are not included in Fig. 3(b).
Then, in Fig. 4, we display the result of applying all discussed theoretical and experimental constraints, limitedly to the reduced interval 0.98 ≤ cos α ≤ 1, since it is the region in which all the analyzed observables converge.Here, we only show the most restrictive bounds so as to not overload the plot.Among the latter, the unitarity bound plays a special role, as it helped us to find a lower limit for the singlet scalar VEV, v s , depending on the Flavon mass, e.g., for M H F = 1000 GeV one has v s ≥ 345 GeV.By comparison, the intersection of all κ i 's and ∆a µ imposes a less stringent upper limit of v s ≤ 1200 GeV 2 .
As far as the CP-even Flavon mass M H F is concerned, to constrain it, we use the limit on the cross section of the process pp → φ → hh from [77], in which a combination of searches for SM-like Higgs boson pair production in proton-proton collisions at √ s =13 TeV and 35.9 fb −1 is reported.We present in Fig. 5 the cross section of the process σ(pp → H F → hh) in the FNSM as a function of M H F and its comparison with the limit on σ(pp → Φ → hh), where φ stands for 1 Such a choice was adopted in the evaluation of κ τ τ and κ µµ , respectively, and then we scanned on the cos α − v s plane, as shown in Fig. 3(a). 2 Notice that, to generate Figs.3(a), 3(b) and 4, we have used our own Mathematica package, so-called SpaceMath [87], which is available upon request.
a generic spin-0 resonance.Furthermore, we show in Figs.6(a) and 6(b) a comparison between the FNSM predictions and the ATLAS Collaboration limits [88], now for individual channels with final states b bb b and b bτ − τ + , respectively.The most stringent constraints [89] come from b bγγ production channel as shown in Fig. 7.In obtaining such limits, we have evaluated the inclusive cross section of our signal process, wherein we have used v s = 1000 GeV ans cos α = 0.995.It is observed that the M H F = 300 − 1000 GeV interval satisfies the bounds imposed, so we will define Benchmark Points (BPs) with H F masses herein.The model parameter space in this analysis is also consistent from the other search channels pp → H F → ZZ at ATLAS [13] and pp → H F → W W at CMS [90].
Our Model:σ(pp→HF )Br(HF →hh) ATLAS:Expected limit (95% CL) ATLAS:Observed limit (95% CL) ATLAS:Expected limit ±1σ FIG.7: Upper limits (observed and expected) on the cross section for di-Higgs production [89] through an intermediate heavy particle φ as a function of the particle mass M φ as obtained through the process pp → H F → hh (h, → b b, h → γγ).

D. Constraints on Ztc from flavor-violating Higgs decays
Finally, because the g H F tc coupling is proportional to the Ztc matrix element, we need a bound on it in order to evaluate the H F → tc decay.Currently, there no specific processes that provide a stringent limit Ztc , but we can estimate its order of magnitude by considering the upper limit on the Branching Ratio (BR) of t → ch at < 1.1 × 10 −3 [91].We also consider the prospects for BR(t → ch) < 4.3 × 10 −5 searches at the FCC-hh [92].The resulting allowed region in the v s − Ztc plane is illustrated in Fig. 8.It is worth noting here that the behavior of the Ztc matrix element shows an increasing (decreasing) trend as v s increases (decreases).This observation is expected since the g htc coupling is governed by Ztc /v s .In order to have a realistic evaluation of the observables studied here, we adopt conservative values for Ztc and v s .FIG. 8: Allowed region in the v s -Ztc plane from the current bound on BR(t → ch) < 1.1 × 10 −3 (blue color) and the projection at the FCC-hh (orange color).

IV. COLLIDER ANALYSIS
Following our discussions on various model parameters and their constraints, we now study the collider signature emerging in the FNSM in the form of a singlet-like CP-even heavy Higgs scalar H F decaying into SM-like Higgs h, neutral gauge bosons Z and top-charm quark pairs at Run 3 of the LHC as well as the HL-LHC, assuming √ s = 14 TeV for both and a luminosity of 3000 fb −1 .In our analysis, we adopt c α = 0.995 (i.e., a small mixing angle α between the CP-even part of the doublet and singlet scalar fields) and assume for the cut-off scale Λ F = 10 TeV, in order to easily avoid theoretical as well as experimental bounds (as discussed in the previous section).Specifically, at the LHC, we consider the resonant production of the H F state via gluon-gluon fusion, followed by its decay into two on-shell SM-like Higgs bosons (h), neutral gauge bosons Z and a top-charm quark pair.For hh production, one of the Higgs h decays into a pair of b-tagged jets while the other decays into two photons, i.e., pp → H F → hh (h → b b, h → γγ): recall Fig. 1.For the ZZ channel, a Z decays into a SFOS pair; while for tc channel, the top quark decays into ν b, with = e − , e + , µ − , µ + .Hence, we have three separate final states.The first one has two photons (γ) and two b-jets, the second one has four leptons, and the third one contains a charged lepton plus its corresponding neutrino and two jets (one of them is a b-jet and the other is a c-jet).They all have some amount of hadronic activity generated from the initial state.Here, we only analyze the channels H F → hh, ZZ, tc, since it is to be noted that the A F hh and A F ZZ couplings are zero because of CP conservation, hence the twin production processes pp → A F → hh, ZZ via gluon-gluon fusion is not possible.The A F → tc decay is dedicated for future analysis.
We use FeynRules [93] to built the FNSM model and produce the UFO files for MadGraph-2.6.5 [94].Using the ensuing particle spectrum into MadGraph-2.6.5, we calculate the production cross section of the aforementioned production and decay process.The MadGraph aMC@NLO [94] framework has been used to generate the background events in the SM.Subsequent showering and hadronization have been performed with Pythia-8 [95].The detector response has been emulated using Delphes-3.4.2 [96].The default ATLAS configuration card which comes along with the Delphes-3.4.2 package has been used in the entirety of this analysis.For both the signal and background processes, we consider the Leading Order (LO) cross sections computed by MadGraph aMC@NLO, unless stated otherwise.FIG.9: The red (blue and purple) line on the left plot stands for the cross section of the processes pp TeV.The variation in the BRs of the heavy CP-even Flavon mass M H F is displayed in the right plot.The heavy Higgs Flavor-violating decay is absent here, i.e., Zij = 0 (i = j).
In the previous processes, we focus on the complete F N diagonal basis, meaning no heavy Higgs Flavor-violating decay is present.This choice allow us to explore the large BRs to other channels, which could potentially provide a large signal significance in our study.We discuss the details now.Afterwards, we consider the F N off-diagonal basis to have new signals.This modification enables us to investigate the effects of heavy Higgs Flavor-violating interactions, which can have significant implications for our understanding of the F N Higgs sector.
We first generate the signal events for various heavy CP-even Flavon masses, M H F (= M A F ) considering Zij = 0 (i = j).The latter have been varied from 260 to 1000 GeV with a step size of 10 GeV.We then take v s = 1000 GeV: such a large VEV produces a small production cross section σ(pp → H F ) and a correspondingly small partial width Γ(H F → hh, ZZ), hence small (but non-negligible, for our purposes) signal rates, however, this is necessary to comply with all theoretical and experimental limits.We display the cross section of the process pp → H F , pp → H F → hh (h → γγ, h → b b) and pp → H F → ZZ (Z → ¯ ) on the left-hand-side of Fig. 9, where the red line stands for σ(pp → H F ).
One can thus understand the nature of the production and decay rates as follows.The production cross sections of the heavy CP-even Flavon H F (or pseudo scalar A F , for that matter) mainly depends on the g 2 ) coupling, as the latter goes into the effective Higgs-to-two gluon vertex, hgg.The corresponding term in the Lagrangian is given by [97]: In this model, the ggh, ggH F and ggA F couplings take the following form: g hgg = cαvs−sαv vs g hgg , g H F gg = cαv+sαvs vs g hgg and g A F gg = v vs g hgg , respectively.It is to be noted that, for Hence, one can understand the shape of the plot by exploiting these functions.The BRs of H F into various channels for v s = 1000 GeV are shown on the right-hand-side of Fig. 9. From the BR plot, we can see that, for heavier H F masses, this state dominantly decays into t t.For small masses, H F → W W dominates.Yet, H F → hh is the third, while H F → ZZ is the fourth largest decay channels.In the next subsections, we will focus on discussing the processes H F → hh (h → b b, h → γγ) and H F → ZZ (Z → ¯ ) for the diagonal and H F → tc (t → b ν ) for off-diagonal scenario, respectively.These processes are of particular interest because they are not as strongly suppressed by standard model backgrounds compared to the H F → t t and H F → W W decays.
The major SM backgrounds typically have the form hh + X (where X is known SM particles), which includes SM Higgs pair hh production, h + X like hZ, hb b and ht t, as well as the non-Higgs processes which include t t and t tγ (here, leptons may fake as photons) as well as b bγγ, ccγγ and jjγγ (where c-jets and light-jets may fake b-jets).The other relevant reducible backgrounds comprise b bjγ, ccjγ and b bjj, where c-jets may appear as b-jets and a light-jet may fake a photon.The fake rate of a light-jet j into a photon depends on the momentum of the jet, p j T [98], as 9.3 × 10 −3 exp(−p j T /27.5 GeV).The c-jet is misidentified as a b-jet with a rate of 3.5% whereas a light-jet mimics a b-jet with a rate of 0.135% [99].

BPs [GeV]
The other input parameters BP1 (M

TABLE II:
The input parameters of the three BPs (BP1, BP2 and BP3) used in the remainder of the paper.We have M h = 125.5 GeV, cos α = 0.995, v s = 1000 GeV and Λ F = 1 TeV is this kept fixed for all BPs.
We next present a detailed discussion of the collider search strategy employed to maximize the signal significance in the search channel pp → H F → hh (h → γγ, h → b b).To start with, though, we show the production and decay cross section pp   IV: The cross sections for the most relevant SM background processes.(Note that these background rates will be multiplied by the fake rates during the analysis.)the three BPs presented in Tab.II (with, in particular, M H F = 800, 900 and 1000 GeV, as seen in Tab.III).The corresponding dominant SM backgrounds are shown in Tab.IV.

BRs and cross sections [pb] BR(H
Any charged objects (leptons or jets) or photons produced in any hard scattering process at the LHC will be observed in the detector if and only if they satisfy certain geometric criteria, known as acceptance cuts.These are the same for both the signal and background events and reproduce the accessible region of the detector.We will then have to ask that both signal and background events pass these acceptance cuts, which are, in general, not sufficient to separate the two samples.However, eventually, we will construct various kinematic observables and study their distributions.Next, we will decide the final selection cuts after studying the distinguishing features of those distributions between signal and backgrounds, so as to increase the former and decrease the latter.We base this approach on a Monte Carlo (MC) analysis using the tools previously described.
In our current scenario, an event is required to have exactly two b-tagged jets and two isolated photons (γ) in the final state.However, we do not put any constraints on the number of light-jets.We then adopt the following acceptance cuts: After considering these basic requirements, we apply a stronger selection (using additional kinematic variables) in order to enhance the signal-to-background ratio, as explained.A variety of such observables have been used to design the optimized Signal Region (SR), i.e., where the significance is maximized.First and foremost, the transverse momentum of photons (p γ 1 T , p γ 2 T ) and b-jets (p b 1 T , p b 2 T )3 will be studied.In addition, the separation between the two final state photons ∆R γ 1 γ 2 and b-jets ∆R b 1 b 2 are also used.The separation between two detector objects, ∆R, is defined as ∆R = ∆η 2 + ∆φ 2 , where ∆η and ∆φ are the differences in pseudorapidity and azimuthal angle, respectively.Then, the invariant mass of the final state photons (M γ 1 γ 2 ) and b-jets (M b 1 b 2 ) will also be used to discriminate between signal and backgrounds, where we Finally, we use the invariant mass M hh for the final extraction.The M hh variable has been calculated as In the above formulae, E and p i (i = x, y, z) stand for the energy and three-momentum component of the final state particles, respectively.FIG.11: Normalized distributions in b-jet transverse momentum for signal and total background after the acceptance cuts.
The (arbitrarily) normalized distributions of all these kinematic variables for the three signal BPs and the total background are shown in Figs.10-14.Based on their inspection, as intimated, we then perform a detailed cut-based analysis to maximize the signal significance against the background.The sequence of constraints adopted is shown in Tab.V. Specifically, notice that, in applying the last requirement herein (on the M hh variable), one may assume that the M H F value is a trial one, if it were not already known from previous analysis.
The signal yields for BP1, BP2 and BP3, along with the corresponding background ones, obtained after the application of the acceptance and selection cuts defining the SR, are shown in Tab.VI for √ s = 14 TeV and, e.g., L = 3000 fb −1 .We initially calculate the signal significance using the relation σ = S √ S+B .Here, S and B stand for the Signal and (total SM) Background rates, respectively.The number of S and B events is obtained as S, B = Aσ S,B L, where and A stand for the selection and acceptance cut efficiency, respectively, σ S,B is the S or B cross section and L is the luminosity.Based on these definitions, it is clear from Tab. VI that strong HL-LHC sensitivity exists for all M H F choices, ranging from discovery (at small masses) to exclusion (at high masses).(It should be appreciated that these significances would be reduced by as much as 30% in the absence of the final M hh selection.)In fact, one can also consider the systematic FIG.12: Normalized distributions in di-photon and di-jet separation for signal and total background after the acceptance cuts.5% for the latter, the significance in Tab.VI for BP1 decreases to 3.75 while for BP2 and BP3 it becomes 2.31 and 1.24, respectively.Hence, the HL-LHC sensitivity is very stable against unknowns affecting the data sample estimations, whatever the origin.We now derive the various projected limits over the M H F − v s plane.It is to be noted that the variation of the singlet scalar VEV v s will directly change the H F hh coupling and correspondingly the production cross section σ(pp → H F → hh).In particular, the smaller the former the larger the latter.To accurately delineate sensitivity regions, we generate a large number of signal events for various combinations of heavy CP-even Flavon mass, M H F , and singlet scalar VEV, v s .Specifically, M H F (≡ M A F ) has been varied from 800 GeV to 1000 GeV with a step size of 5 GeV while v s has been varied between 500 and 1000 GeV with a step size of 25 GeV.The projected exclusion (2σ) region derived from the γγb b final state in the M H F − v s plane are given in Fig. 15.The left plot is drawn for L = 3000 fb −1 (HL-LHC).Again, the left plot in Fig. 15 is shown with no systematic uncertainty, i.e., κ = 0, while the right plot is drawn based on a systematic uncertainty κ = 5%.From the right plot, we should mention that the limits drop somewhat (by 5 − 10%) upon introducing a systematic uncertainty of κ = 5%, hence not too drastic a reduction of sensitivity in general (as already remarked for our BPs).

Benchmark points: Signal and Significances BP1 (M H
In this section, we now discuss the signatures involving the final state with four leptons (2 ¯ +2 ) in the context of HL-LHC.The primary contribution to these signatures typically arises from the process pp → H F → ZZ, where each Z boson further decays into a lepton-antilepton pair (Z → ¯ ).To investigate the leptons' final state signatures, we have selected the same three benchmark points, which are M H F = 800, 900, and 1000 GeV, respectively.The table VII displays the signal cross-sections for different processes.Among them, the primary background in the Standard Model is the production of two Z bosons accompanied by jets (ZZ + jets).In addition, there are other significant reducible backgrounds, such as the production of top quark pairs with jets (t t + jets), the production of a Z boson and a Higgs boson with jets (Zh + jets), and so on.We have included all the relevant Standard Model backgrounds in the table VIII.
In this particular scenario, the event must contain precisely four isolated leptons, consisting of   The matched cross sections for the most relevant SM background processes.(Note that these background rates will be multiplied by the fake rates during the analysis.) two positively charged leptons and two negatively charged leptons.This requirement ensures the presence of same-flavor opposite-sign (SFOS) leptons (electron and/or muon) in the final state.However, no specific constraints are imposed on the number of light jets present in the event.We then adopt the following acceptance cuts: • p γ T > 20 GeV; • p e/µ T > 20 GeV; • p j T > 40 GeV, where j stands for light-jets as well as b-jets; After considering these basic requirements, we apply additional cuts using kinematic variables to enhance the signal-to-background ratio.Various such kinematic variables have been used to design the optimized Signal Region (SR), i.e., where the significance is maximized.First and foremost, the transverse momentum of the leptons (p i T , i = 1..4) and the minimum invariant mass M min out of four combinations (M i j , i, j = 1..4) and total transverse momentum of four leptons ( p i T ) will be studied.
Here E and p i (i = x, y, z) stand for the energy and three-momentum component of the final state leptons, respectively.
The normalized distributions of all these kinematic variables for the three signal BPs and the total background for this analysis are shown in Figs.16-17.We then perform a detailed cut-based analysis to maximize the signal significance against the SM backgrounds.The figures labeled 16 to 17 illustrate the normalized distributions of various kinematic variables for the three signal benchmark points (BPs) as well as the total background in this analysis.Subsequently, we employ a thorough cut-based analysis technique to optimize the signal significance with respect to the Standard Model backgrounds.The specific sequence of cuts applied during this analysis is presented in Tab.IX.FIG.17: Invariant mass of two leptons, four leptons, sum of all momentum distributions for signal and total background after the acceptance cuts.

Kinematic variables and cuts Observable
Value The optimized SR as a function of the H F mass.
The Tab. X shows the signal yields for three benchmark points and the corresponding yields for the SM background.We obtained these numbers after applying acceptance and selection cuts that define the signal region (SR).The calculations were performed for a center-of-mass energy of √ s = 14 TeV and an integrated luminosity of L = 3000 fb −1 .We calculate the signal significance using the formula σ = S √ S+B , where S represents the signal yield and B represents the background yield.The presence of non-zero Ztc allows for processes such as H → tc, where the heavy Higgs decays into a top quark and an anti-charm quark, or a charm quark and an anti-top quark, respectively.These flavor-violating decays are possible due to the mixing between the top and charm quarks induced by the non-zero Ztc .The observation of such flavor-violating decays would have significant implications for our understanding of the F N heavy Higgs sector.It would provide evidence for new physics beyond the SM, as the SM predicts negligible flavor violation in the Higgs sector.The presence of flavor-violating decays would suggest the existence of new particles or interactions that can induce such processes.
Studying the properties of the flavor-violating decays, such as their rates and kinematic distributions, can provide valuable information about the underlying physics responsible for the F N heavy Higgs sector.It can help constrain the model's parameter space and provide insights into the flavor structure and dynamics of the theory.We present the analysis for the production of the H F via proton-proton collisions pp → H F , followed by the FCNC decay H F → tc (t → b ν ) in the presence of non-zero Ztc .The model parameter values used in the simulation are shown in Table XI   The corresponding cross sections for the benchmark points used in this paper are presented in Table XII.Meanwhile, the BR(H F → tc) as a function of the singlet VEV v s and the Ztc matrix element is shown in Fig 18 .We observe BRs(H F → tc) quite large O(0.1), which comes because the couplings H F W W and H F ZZ are suppresed, which allows the opening of the tc channel.
In this analysis, the main SM background comes from the final state of bj ν , whose source arises mainly from W jj + W b b, tb + tj.Another important background is t t production, where either one of the two leptons is missed in the semi-leptonic top quark decays, or two of the four jets are missed when one of the top quarks decays semi-leptonically.The cross sections of the dominant SM background are shown in Table XIII.

SM backgrounds
Cross section [pb] Fig. 19 shows the kinematic distributions generated both by the signal (for M H F = 800 GeV, v s = 1000 GeV) and background processes, namely, the transverse momentum of the particles The following acceptance and kinematic cuts imposed to study possible evidence of the H F → tc (M H F = 800 GeV) at the LHC are as follows.
• We requiere two jets with |η j | < 2.5 and p j T > 30 GeV, one of them is tagged as a b-jet.
• Since an undetected neutrino is included in the final state, we impose the cut MET> 40 GeV.21 displays the contour plots of the signal significance as a function of the integrated luminosity L int and the singlet scalar VEV v s , for M H F = 800, 900, 1000 GeV.Once L int = 300fb −1 of accumulated data is achieved and assuming v s = 640 GeV (625 GeV, 620 GeV), we find that the LHC would have the possibility of exploring a detectable Flavon H F of mass 800 GeV (900 GeV, 1000 GeV).Even more promising results could be found in the HL-LHC era, which could corroborate the possible findings of the LHC regarding the H F → tc process.The hierarchical structure and peculiar pattern of quark and lepton masses in the SM have been a long standing issue coined as the 'flavor puzzle'.Various interesting beyond the SM proposals have been suggested to resolve this riddle.Among these, the one by Froggatt and Nielsen is arguably one of the most fascinating ones.Herein, the scalar sector predicts one singlet complex scalar S F which is charged under a new U (1) F flavor symmetry (which is softly broken).After EWSB and U (1) F breaking, the mixing between the SM Higgs doublet with the real part of the S F singlet produces two physical scalars, h and H F , where h is identified as the SM-like Higgs boson (discovered in 2012) while H F is an additional CP-even (so-called) Flavon with mass O(1 TeV).(The imaginary part of S F is identified as the CP-odd heavy Flavon A F .)The (pseudo)scalar sector of this model is controlled by two parameters: the Flavon VEV v s and the mixing angle α.The structure of various Yukawa couplings of this model is such that one can have FCNCs involving the two new heavy (pseudo)scalars (H F &A F ) even at tree-level.The corresponding contributions to FCNC processes thus attract severe constraints from various low energy flavor physics data.Therefore, in our analysis of such a scenario, we have considered all possible experimental (as well as theoretical) limits on the model parameters v s and α.With the LHC currently running at CERN, it is very tempting to utilize the ongoing (Run 3) and future (HL-LHC) stages of the machine to explore the signature of such heavy flavons.
In this paper, our primary focus was on the CP-even heavy Flavon denoted as H F .We explored its discovery potential at the LHC by investigating its production through gluon-gluon fusion followed by its subsequent decays.We considered various decay modes for it, including into two SM Higgs bosons and two SM (neutral) gauge bosons.By studying these different decay channels and considering the corresponding signatures at HL-LHC (with √ s = 14 TeV), assuming a luminosity of 3000 fb −1 , we were able to confirm the discovery potential (5σ) of the CP-even heavy Flavon H F at the LHC through these SM signatures.In addition, we explored the flavorchanging H F → tc decay, specific to our model, which is predicted to arise when M F m t .This decay can be as large as O(0.1) because the H → V V (V = W, Z) decays are heavily suppressed once the tc channel is opened.We thus showed that this non-SM channel offers an alternative opportunity to test our model, even at the standard LHC.Once a integrated luminosity of 300 fb −1 is reached, we find in this channel a signal significance of up to 6σ for masses of the Flavon between 800 − 1000 GeV.
We have obtained such results following a thorough numerical analysis emulating both the aforementioned signal and the most relevant (ir)reducible backgrounds accounting for hard scattering, parton shower, hadronization and detector effects.We thus advocate that the experimental collaborations at the LHC, specifically, the multipurpose ones (ATLAS and CMS), tackle this search, as its results can lead to a better understanding of the origin and solution of the flavor puzzle in the SM.This should be facilitated by having implemented the advocated model in standard computational tools, which are available upon request.

FIG. 3 :
FIG. 3: VEV of the FN singlet v s as a function of the cosine of the mixing angle α: constraints are from (a) the SM-like Higgs boson coupling modifiers and (b) flavor observables (as described in the text).

FIG. 4 :
FIG.4: VEV of the FN singlet v s as a function of cosine of the mixing angle α in the presence of the most stringent ones among all theoretical and experimental constraints considered.

FIG. 6 :
FIG. 6: Upper limits (observed and expected) on the cross section for di-Higgs production through an intermediate heavy particle φ as a function of the particle mass M φ as obtained through the processes pp → H F → hh (h, → b b, h → b b) (left) and pp → H F → hh (h, → b b, h → τ + τ − ) (right).

>
FIG.10: Normalized distributions in photon transverse momentum for signal and total background after the acceptance cuts.

√
S+B for BP1, BP2 and BP3 corresponding to the optimized SR are shown.In addition, the total background yield and the total signal yield are also given at √ s = 14 TeV with integrated luminosity L = 3000 fb −1 .sum of √ S + B and use σ b = κB [100], i.e., σ = S √ S+B+(κB) 2 , with κ being the percentage of systematic uncertainty of the total background.

1 2σFIG. 15 :
FIG.15:The projected exclusion (light blue) and discovery (dark blue) regions in the M H F − v s plane.These plots are drawn for L = 3000 fb −1 .The right plot is drawn considering a systematic uncertainty κ = 5%.

FIG. 16 :
FIG.16: Transverse momentum distributions for signal and total background after the acceptance cuts.

FIG. 19 :
FIG. 19: Normalized transverse momentum distributions associated to the top decay: (a) leading b-jet, (b) leading charged lepton, (c) tranverse missing energy due to undetected neutrinos; (d) transverse momentum distribution of the c-jet.

Fig.
Fig.21displays the contour plots of the signal significance as a function of the integrated luminosity L int and the singlet scalar VEV v s , for M H F = 800, 900, 1000 GeV.Once L int = 300fb −1 of accumulated data is achieved and assuming v s = 640 GeV (625 GeV, 620 GeV), we find that the LHC would have the possibility of exploring a detectable Flavon H F of mass 800 GeV (900 GeV, 1000 GeV).Even more promising results could be found in the HL-LHC era, which could corroborate the possible findings of the LHC regarding the H F → tc process.

FIG. 21 :
FIG. 21: Contour plots for the signal significance as a function of the integrated luminosity and the singlet VEV v s .(a) M H F = 800 GeV, (b) M H F = 900 GeV, (c) M H F = 1000 GeV.In these results we consider a systematic uncertainty κ = 5%

TABLE I :
Tree-level couplings of the SM-like Higgs boson h and the Flavons H F and A F to fermion and gauge boson pairs in the FNSM.Here, r s = v/ √ 2v s .

TABLE III :
The BR(H F → hh) and cross sections for the processes pp → H F and σ(pp → H F → hh, h → γγ, h → b b) for three BPs (BP1, BP2 and BP3) used in the remainder of the paper.

TABLE VII :
The BR(H F → ZZ) and cross sections for the processes pp → H F and σ(pp → H F → ZZ, Z → ¯ ) for three BPs (BP1, BP2 and BP3) used in the remainder of the paper.

TABLE X :
The signal significance σ = S √ S+B for BP1, BP2 and BP3 corresponding to the optimized SR are shown.In addition, the total background yield and the total signal yield are also given at √ s = 14 TeV with integrated luminosity L = 3000 fb −1 .

TABLE XI :
Model parameter values used in the Monte Carlo simulation.FIG.18: BR(H F → tc) as a function of the singlet VEV v s and the Ztc matrix element.

TABLE XII :
The BR(H F → tc) and cross sections for the processes pp → H F and σ(pp → H F → tc, t → ν b) for three BPs (BP1, BP2 and BP3) used in the remainder of the paper.

TABLE XIII :
Cross section of the dominant SM background processes.