Lepton masses and mixing in a two-Higgs-doublet model

Within the framework of the two-Higgs Doublet Model (2HDM), we attempt to find some discrete, non-abelian flavour symmetry which could provide an explanation for the masses and mixing matrix elements of leptons. Unlike the Standard Model, currently there is no need for the flavour symmetry to be broken. With the GAP program we investigate all finite subgroups of the U3 group up to the order of 1025. Up to such an order there is no group for which it is possible to select free model parameters in order to match the masses of charged leptons, masses of neutrinos, and the Pontecorvo-Maki-Nakagawa-Sakata mixing matrix elements in a satisfactory manner.


I. INTRODUCTION
Despite the success of the Standard Model (SM) in providing a description of the current experimental data, there is a widespread belief that sooner or later an increase in available energy or accuracy of measurements will lead to the detection of a discrepancy between experimental results and theoretical predictions. The SM cannot be considered a complete theory because it does not provide answers to many pressing questions. One of the most important issues to be resolved concerns the masses of the fundamental constituents of matter, quarks and leptons. At the moment we still cannot theoretically predict their masses; we are only able to obtain their values from experimental data.
The discovery of the Higgs particle offers a partial solution to the problem, yet it does not resolve it completely. Particles acquire their mass by means of interaction with the Higgs field, and rather than examine the numerical values of the masses, we are currently more interested in the question of why particles interact so differently with the Higgs field, or in other words, why the Yukawa couplings cannot be theoretically predicted.
The solution to the problem of the elementary particles' masses is important in itself, because it would reduce the amount of unknown free parameters in the SM. Another important reason for conducting such investigations is that they create a great opportunity to understand the origin of the masses of physical bodies. The main part of the mass of each physical body comes from the interaction of the ingredients contained in it. But it is not the entire mass; the remaining part (though small) being the masses of individual fermions, which are still undetermined.
Several proposals to solve this problem, at least partially, can be found in the literature (see e.g. [1][2][3]). Although the problem concerns all matter con-stituents, here we will concentrate on an attempt to explain the masses and mixing angles of leptons. One of the most common approaches consists in the imposition of a flavour symmetry on the leptonic part of Yukawa Lagrangian (for review see e.g. [4][5][6]). This approach was particularly popular and successful before 2011 when it was discovered that the reactor-mixing angle θ 13 is nonvanishing [7,8]. The models with an additional flavour symmetry are very popular, but they are by no means the only ones some papers have been published in which the very existence of such a symmetry is denied [9,10].
Attempts at solving the problem of lepton masses by a horizontal symmetry are dependent on the manner of the introduction of neutrino masses as well as on the Higgs sector for spontaneous symmetry breaking. In the simplest case of the conventional SM with one Higgs doublet in which neutrinos are massless, only three additional right-handed neutrinos are introduced. In such an extension of the SM, without introducing the Majorana term, neutrinos are Dirac particles [11]. It is not necessary to introduce the right-handed neutrinos to obtain their masses. Instead, it is possible to use the existing left-handed neutrinos to form the Majorana masses [11]. Both cases will be considered in this paper.
Within the framework of the Standard Model with one Higgs boson, a discrete symmetry for Yukawa couplings provides the relations for the three-dimensional mass matrices of charged leptons (M l ) and neutrinos (M ν ) [12,13]: where A i L = A L (g i ), i = 1, 2, . . . , N are three dimensional representation matrices for the left-handed lepton doublets for some N-order flavour symmetry group G.
In such a case, the I-st Schurs lemma implies that M l M † l and M ν M † ν are proportional to the identity matrices, which clearly entails the trivial lepton mixing matrix (known in the literature as Pontecorvo-Maki-Nakagawa-Sakata (PMNS) matrix [14][15][16]).
In order to avoid the relations given in (2) and predict the non-trivial lepton masses and their mixing in this case, the family symmetry has to be broken. As a rule, the flavour symmetry G is spontaneously broken by scalar singlet Higgs fields called flavons (see e.g. [5,17,18]). However, it can also be broken by introducing a bigger number of normal Higgs multiplets (e.g. [18,19]). The latter way is indeed more economical, since the spontaneous gauge symmetry breaking in this case gives rise to the particles masses, simultaneously leading to the break of a family symmetry. Additional flavon scalar fields in this framework are therefore redundant.
Such models were considered many times in the literature, but mostly in the frame of supersymmetric models where Higgs doublets were singlets of a flavour group [20,21], or, in a more general approach, where only one selected flavour group was tested [22][23][24].
In the present work we attempt to explore how the Two Higgs Doublet Model works in the context discussed above. As distinct from the previous works, we do not consider a few selected discrete groups, but instead we try to find a flavour symmetry in all groups up to 1025 order with one restriction, i.e. each of our groups must have at least one faithful, three dimensional irreducible representation.
In the next Section, we briefly introduce the flavour symmetry in the 2HDM model and show how the symmetry transformation between two Higgs doublets provides an opportunity to avoid the consequences of Schur's lemma. We also present all the formulas needed to conduct the computations in the case of Dirac and Majorana neutrinos. In Sec. III the results of the final scan of the Yukawa matrices, the lepton masses and the PMNS mixing matrix elements are presented, and finally in Sec. IV we draw our conclusions.

A. Dirac neutrinos
To begin, the leptonic part of the Yukawa Lagrangian with Dirac neutrinos will be considered. In contrast to the Standard Model, two Higgs doublets Φ i contribute to the lepton masses (the so-called Two-Higgs-Doublet-Model of type III [25]) as follows [26] where: are gauge doublets for the left-handed lepton and Higgs fields and the fields l βR , ν βR stand for the right-handed lepton and neutrino fields, respectively. The couplings h create the 3-dimensional Yukawa matrices. The spontaneous gauge symmetry breaking gives nonzero vacuum expectation values (VEVs) v i for the Higgs doublets: and the mass matrices read as follows [26]: In general the vacuum expectation values can be complex, v i = |v i |e iϕi , but they are restricted by the Fermi coupling constant: Family symmetry of our theory implies, that after the transformation of fields occurring in the 2HDM Lagrangian by the 3 dimensional (A L , A R l , A R ν ) and 2 dimensional (A Φ ) representations of a flavour group G: the full 2HDM Lagrangian does not change: Given that all the transformation matrices in Eq.(8) are unitary, the only parts of the total 2HDM Lagrangian for which the aforementioned relations are not automatically fulfilled are the Yukawa Lagrangian and the Higgs 1 Note that, in comparison to our notation, in the paper [26]: potential. Imposition of symmetry on these terms of the model severely restricts their forms. The invariance of the Yukawa Lagrangian is expressed as follows: With regard to the Higgs potential, there appear to be two possibilities. The first one assumes that: which implies that before and after the transformation for Higgs fields (Eq. (8)) the coefficients in the potential remain exactly the same and the vacuum expectation values are equal, v ′ i = v i . There is also a second possibility, useful for phenomenological reasons, we allow for the modification of VEVs, which transform in the same way as the Higgs fields: In this case, the form of the Higgs potential does not change, the terms in the potential do not vary, while only the potential coefficients undergo change. This kind of invariance is known in the literature as the form-invariance (see [26,27]). After the unitary transformation (Eq. (12)), the condition given in Eq. (7) is unchanged, and hence also: The vacuum expectation values, whose sum of squares is constant, will need adjustments to meet the experimental requirements. Thus, from the point of view of flavour symmetry, the type of Higgs potential is irrelevant, so in our approach, the issue of what symmetry for Higgs Lagrangian is chosen becomes insignificant. With reference to the Eq. (9), in order to find symmetric Yukawa matrices h i , i = 1, 2, one can readily express the symmetry conditions as the eigenequation for a direct product of unitary group representations to the eigenvalue 1 (see e.g. [28]): for the charged leptons and neutrinos, respectively. Both relations (Eq. (14)) need to be satisfied for any group's element g ∈ G. It is however sufficient that they are fulfilled only for the group generators [28], which considerably reduces the time of the computation.
In such a model, the invariance equations for the mass matrices are not trivial. For the symmetric Higgs potential (Eq.(11)) : (15) then Eq.(1-2) are not satisfied and we avoid the consequences of the Schur's Lemma. The same can be shown for the form-invariant Higgs potential, where Eq.(12) is satisfied. In this context we can obtain the non-trivial mass matrices without the introduction of additional flavon fields.

B. Majorana neutrinos
For Majorana neutrinos the Yukawa term has to be changed. In 2HDM, the simplest Yukawa Lagrangian can be taken as the non-renormalizable Weinberg term in the form: where L c βR = CL T βL is the charge conjugated lepton doublet fields. After the spontaneous symmetry breaking, the neutrino mass matrix is obtained: As in the preceding case, in compliance with the requirement of flavour symmetry for the Yukawa Lagrangian (Eq. (16)), the neutrino Yukawa matrices must satisfy the eigenvalue equation: The flavour group G F , which is imposed on the 2HDM Lagrangian, cannot be arbitrary. Due to the fields' transformations defined in Eq.(8), the group must possess at least one 2-dimensional (for A Φ ) and at least one 3dimensional irreducible representation (for A L , A R ν and A R l ). The sole application of irreducible representations is justified given the fact that were any of the representations (A Φ , A L or A R ν (A R l )) to be reducible, invariant Yukawa couplings would split up into independent sets for irreducible representations (see e.g. [28]).
In the selection of flavour symmetry groups, we have limited ourselves to finite-dimensional groups of the order of at most 1025, which are furthermore subgroups of the U (3) group (at least one of the 3-dimensional irreducible or reducible representations must be faithful) [29,30]. This additional condition is not necessary [31] but it significantly reduces the number of groups to be processed. Using the GAP (version 4.7.6) [32] system for computational discrete algebra with the included Small Groups Library [33] and REPSN [34] packages, we have found in total 10862 groups with at least one 2-dimensional and at least one 3-dimensional irreducible representation, but only 413 of these groups are subgroups of the U (3) group. They split into two disjoint sets. Each group has either at least one faithful 3-dimensional irreducible representation (there are 173 such groups), or at least one faithful 1+2 reducible representation (there are 240 such groups). Some groups are also subgroups of the U (2) group. They have at least one faithful 2-dimensional irreducible representation (none of them have any faithful 3-dimensional irreducible representation). None of the groups have any faithful 1+1+1 reducible, faithful 1+1 reducible or faithful 1-dimensional irreducible representation. All the groups which delivered any solutions belong to polycyclic groups that use the polycyclic presentation for element arithmetic (so called P C-groups).
All the detected groups of the order of at most 1025 are listed in the ancillary file, while Table I shows groups of the order of at most 100 only which underwent investigation.

B. Yukawa couplings matrices in the model with Dirac neutrinos
In the case of all the groups subject to consideration, there exist 267 groups that gave in total 748672 different combinations of 2-and 3-dimensional irreducible representations that give 1-dimensional degeneration subspace for all generators, which is the solution to the equations in Eq. (14). This common vector gives Yukawa matrices for charged leptons (h (l) ) and for neutrinos (h (ν) ), which are interrelated. All the possible solutions for Yukawa matrices for charged leptons and for Dirac neutrinos can be expressed through 7 base forms (ω = e 2πi/3 ): the next three h Symmetric Yukawa matrices for any considered group and for any irreducible representation within the group can be expressed now by seven (i=1,2,..,7) basic matrix forms as follows: a) for Dirac neutrinos: b) for charged leptons:  [ where ϕ is a phase distinctive for a group and for irreducible representations and δ l = 0, π. In order to find the lepton masses and mixing matrix we with k = −1, 0, +1 and κ = |v 2 |/|v 1 |, the same for neutrinos and for charged leptons. The only difference lies in the phase η x . For Dirac neutrinos, and for charged leptons, where ϕ i (i = 1, 2) are phases of the VEVs v i . After diagonalization of Eq.(25) we obtain: and the diagonalization matrix U: This matrix does not depend on the phase η x , so it is identical for charged leptons and for the neutrino. Therefore, it is not possible to reconstruct the correct mixing matrix. For groups and their irreducible representations for which δ l = 0, neutrinos and lepton masses are proportional and the mixing matrix is 3x3 identity matrix. For groups and for representations where δ l = π, masses of charged leptons and neutrinos are not proportional. If the formulas in Eqs. (26)(27)(28) describe the masses of neutrinos, then for the charged leptons there is: Similarly in this case we cannot reconstruct the PMNS mixing matrix for which the 3x3 anti-diagonal identity matrix is obtained. Regardless of whether the masses of charged leptons are described by formula Eqs. (26)(27)(28)) (groups with δ l = 0) or by Eq.(30-32) (groups with δ l = π), three parameters (c l , κ, η x ) cannot be selected in such a way as to obtain the physical masses of the electron, muon and tau (in any case, one obtains: 0 m e /m µ 1 and 1 m τ /m µ 2).
The SM extended by one additional doublet of Higgs particles (2HDM) does not posses a discrete family symmetry (in the groups under examination) that can explain the masses of charged leptons, masses of neutrinos having the nature of Dirac particles and the PMNS matrix.

C. Yukawa couplings matrices in the model with Majorana neutrinos
In the previous subsection, we did not receive masses of charged leptons consistent with the experiment. This was the case when the neutrinos were Dirac particles. In the case of Majorana neutrinos, we must once again look for a possible symmetry, since a symmetric solution for the Weinberg component may also deliver other groups of symmetry for charged leptons. We are currently looking for symmetries satisfying the Eg. (14) for the charged leptons and Eq.(18) for neutrinos. Out of all the groups under consideration, there exist 195 groups that gave in total 20888 solutions. All the found symmetries, as a solution of Eq.(18), give a two dimensional space which is common to all generators of the groups in question. Each time we find two 36-dimensional vectors p and r and any linear combination of p and r, gives symmetric Yukawa matrices for Majorana neutrinos. Thus these Yukawa matrices are given by: where x and y are two free complex numbers. A detailed analysis of the solution for Majorana neutrinos is as follows: and for the charged leptons Yukawa matrices: 2 , e −i(δ l +ϕ) h 1 }, where h 1 , h 2 are given by Eq. (22) and I 3 is 3x3 identity matrix. As previously observed, the phases δ = (0, π), δ l and ϕ depend on the group and its representations.The resulting neutrino mass matrix Eq.(17) has the form; 2 + y|v 1 v 2 |e i(ϕ1+ϕ2) I 3 1 + e iδ + x|v 2 | 2 e i(δ+2(ϕ2+ϕ)) h which gives the squares of neutrino masses: m 2 1 =|c ν | 2 1 + κ 4 + 2β 2 κ 2 + 2β 2 κ 2 cos(δ) + 4βκ 3 cos m 2 2 =|c ν | 2 1 + κ 4 + 2β 2 κ 2 + 2β 2 κ 2 cos(δ) + 4βκ 3 cos m 2 3 =|c ν | 2 1 + κ 4 + 2β 2 κ 2 + 2β 2 κ 2 cos(δ) − 4βκ 3 cos where β = y/x , τ = ϕ 2 −ϕ 1 and c ν = gx|v 1 | 2 e 2iϕ1 /(2M ). The masses of charged leptons are given by the same formulae as given above Eqs. (30)(31)(32) and it is impossible to fit electron, muon and tau lepton masses. Our mass matrices for neutrinos and charged leptons are diagonal, so we also do not have the ability to fit the mixing matrix.
We also see that the symmetry condition for Majorana neutrinos (Eq.18) does not give any new flavour symmetry group with new three dimensional representation A L which would not be present in the symmetry equations for charged leptons (Eq.14). The Yukawa matrices h (l) 1 and h (l) 2 in Eq.35 for charged leptons are exactly the same as in the case of Dirac neutrinos (Eq.24). This is not a good conclusion. In the set of groups we are consider, regardless of the adopted neutrino sector, we will not find a symmetry that gives acceptable solutions for mass of charged leptons.

IV. CONCLUSIONS
We have explored the possibility of using some discrete flavour symmetry to explain the masses and mixing matrix elements of leptons in the Standard Model with the Higgs particle sector extended by one additional Higgs doublet -2HDM. In general, we have assumed that the total Lagrangian model has a full flavour symmetry with one exception. We also admit that the Higgs potential in our model is only invariant in form, which is often used in the model description of experimental data. In such a model, we have avoided having to break the family symmetry and introduce flavon fields. We have investigated discrete groups that are subgroups of the continuous U (3) group up to the order of 1025. Models in which neutrinos have the nature of Dirac particles and models with Majorana neutrinos have been considered. Following a close analysis of these 413 groups and all their possible combinations of 2-and 3dimensional irreducible representations, it is established that none of them can reproduce the current experimental data. Thus, in the 2HDM model with the symmetrical or only form-invariant Higgs potential, in the class of the groups under consideration, there is no discrete family symmetry that would fully clarify the masses and parameters of the mixing matrix for leptons. In addition, we have observed that the set of symmetric Yukawa matrices for charged leptons is independent from the nature of the neutrinos. This can serve as a guidance for further search for family symmetry. In the models with two Higgs particles, regardless of the adopted neutrino sector and the set of groups that we consider, we do not find a symmetry that gives real masses of charged leptons, even approximately.
This work has been supported by the Polish National Science Centre (NCN) under grant no. UMO -2013/09/B/ST2/03382