Recognizing symmetries in 3HDM in basis-independent way

Higgs doublets may come in three generations. The scalar sector of the resulting three-Higgs-doublet model (3HDM) may be constrained by global symmetry groups $G$ leading to characteristic phenomenology. There exists the full list of symmetry groups $G$ realizable in the 3HDM scalar sector and the expressions for $G$-symmetric scalar potentials written in special bases where the generators of $G$ take simple form. However recognizing the presence of a symmetry in a generic basis remains a major technical challenge, which impedes efficient exploration of the 3HDM parameter space. In this paper, we solve this problem using the recently proposed approach, in which basis-independent conditions are formulated as relations among basis-covariant objects. We develop the formalism and derive basis-independent necessary and sufficient conditions for the 3HDM scalar sector to be invariant under each of the realizable symmetry group. We also comment on phenomenological consequences of these results.


I. INTRODUCTION
A. Historical context The scalar potential of the Standard Model (SM) minimally includes a single doublet of SU (2) L which reduces the electroweak symmetry to electromagnetism via the Brout-Englert-Higgs mechanism, see the recent review [1] and references therein. The associated single physical Higgs boson has been observed [2,3] and is now being extensively investigated at the LHC. However whether the Higgs sector is indeed as minimal as postulated by the SM or if the observed 125 GeV Higgs is just the first state of a rich scalar sector is presently unknown. This question can only be answered by experiment. In anticipation of possible future hints or discoveries, theorists investigate other, non-minimal Higgs sectors and look for novel ways to experimentally probe them, see e.g. [4].
A simple and well motivated generalisation of the SM is extending the scalar sector to include further SU (2) L doublets. This can be thought of as bringing to the scalar sector the concept of generations present in the SM fermion sector. Historically, the main motivations for going beyond the minimal scalar sector of the SM were to gain insight into the origin of CP violation (CPV) and into the general flavour puzzle.
In 1973, T. D. Lee suggested that CP can be broken spontaneously in a model with two Higgs doublets (2HDM) [5,6]: one starts with a lagrangian which is explicitly CP -invariant but observes that the vacuum expectation values (vevs) emerging after the scalar potential minimization break the symmetry. However, one typically obtains in this case dangerously large tree-level flavor changing neutral currents (FCNCs). Although they can be eliminated by imposing natural flavor conservation (NFC) [7,8], this extra requirement precludes any CP violation, explicit or spontaneous. This clash was removed by S. Weinberg in 1976 in a model with three Higgs doublets (3HDM) [9] with explicit CPV and later by G. Branco in the spontaneously CP -violating model [10,11]. See also e.g. [22,23] for more possibilities to control FCNCs in N -Higgs Doublet Models (NHDMs).
The late 70's also witnessed a surge of activity on linking the fundamental fermion masses with the entries of the Cabibbo-Kobayashi-Maskawa (CKM) mixing matrix. 3HDMs equipped with discrete symmetry groups offered many intriguing opportunities. In 3HDMs, the number of Higgs doublets matches the number of fermion generations, which is viewed as an appealing feature of the models.
During 1990's and 2000's, exploration of multi-Higgs-doublet model was dominated by 2HDMs, boosted by two Higgs doublets being required in minimal supersymmetric extensions [20]. In the past decade, 3HDMs gradually re-gained interest since in many aspects they are capable of delivering more than 2HDMs. The attractive phenomenological features of 3HDMs include richer scalar spectrum, CPV simultaneously with dark matter candidates [21], geometrical CPV [24,25], a novel type of CP symmetry, which is of order 4 rather than of order 2 [26][27][28] and which is physically distinct from the usual CP [29], and of course a variety of discrete symmetry groups.
Given that 3HDM scalar and Yukawa sectors can be equipped with global symmetries, which have a profound effect on phenomenology, a classification program was undertaken a decade ago to list all symmetry-related situations possible in 3HDMs. First, the list of all abelian symmetries realizable in 3HDM without leading to accidental symmetries was obtained in [30,31] and later extended to Yukawa sectors in [32][33][34]. Next, the full list of all discrete non-abelian symmetry groups realizable in the 3HDM scalar sector was derived in [35,36]. Continuous non-abelian groups were not listed; we will include them in the present work to complete the classification. Finally, a G-symmetric potential can have minima which either conserve or (partially) break the symmetry group. The full list of all symmetry breaking patterns for each group G was presented in [37]. One particularly important conclusion was that, for sufficiently large discrete group G, there remains some residual symmetry in any minimum. In the light of the theorem formulated initially in [38] and refined in [39], this incomplete breaking leads to unrealistic fermion sectors.
B. The challenge of basis independent recognition: the example of CP symmetry Models which involve several fields with equal quantum numbers possess notorious large basischange freedom, which can seriously impede their efficient exploration. Two models may look completely different and in fact correspond to the same physics, merely written in different bases. A model can also contain a symmetry, but if its lagrangian is written in a generic basis, the presence of this symmetry may be obscured. In order to detect the presence of symmetries, one must develop and apply symmetry recognition checks which do not rely on the choice of basis.
The traditional basis-invariant approach to NHDMs with symmetries is best illustrated by the problem of finding necessary and sufficient conditions of explicit CP -conservation in the scalar sector.
In order to understand the properties of the potential under the action of a general CP transformation [40][41][42], one constructs CP -odd invariants (CPI), first identified in [43] and further developed in [44][45][46][47][48][49]. One writes the coupling coefficients of the scalar potential as tensors under the basis change group, then fully contracts these tensors to produce various basis invariant quantities, and selects those invariants which flip sign under the action of a general CP transformation. Although the explicit expression of the general CP transformation is basis-dependent, its action on basis invariants is the same in all bases, and therefore one gets an unambiguous identification of CPIs.
Although there are infinitely many CPIs, there exists a finite number of "generating" CPIs. If all of these generating CPIs are zero, then all other CPIs are also zero, and the model is explicitly CP conserving. One just needs to identify these generating invariants, and this is where the problem becomes difficult.
In the case of 2HDM, the four generating CPIs were established in [46][47][48] with the aid of computer algebra. They were almost immediately derived in a much more transparent way within the bilinear formalism, which appeared first in [50] and which was developed further and applied to CP -conservation in [51][52][53][54][55]. Very recently, the four CPIs of 2HDM were rederived in an alternative approach based on fields rather than bilinears [56].
Extension of these methods to 3HDM turned out very challenging. Although the CPIs can be easily constructed [49], it is unclear how to find the set of generating CPIs. It was done, for example, in simpler cases of 3HDMs with non-abelian symmetries with triplets [57,58], but it remains unsolved in the general 3HDM. Whether the methods of [56] can be generalized to 3HDM and solve this problem remains an open question and requires additional work.
Meanwhile, an alternative approach made its debut in 2006 [59] and was recently exploited fully in [60,61]. The idea is that it is not obligatory to use basis invariants in order to establish basisindependent conditions. One can also formulate these conditions in the form of basis-independent relations among basis-covariant objects [61]. Using this approach, the basis-independent necessary and sufficient conditions were formulated for the usual CP symmetry [59] and for the CP symmetry of order 4 (CP4) [60], as well as for the simultaneous presence of the two forms of CP symmetry.
With these results, the issue of explicit CP conservation in 3HDMs is now settled.

C. Towards basis independent recognition of other symmetries in 3HDM
The "success story" above supports the idea of using basis-covariant objects of the bilinear formalism to detect all other symmetries of 3HDMs. This is what we accomplish in the present work for all the realizable symmetry groups, abelian and non-abelian. The essence of our procedure is the following. We select a symmetry group, write the general Higgs potential invariant under it in a convenient basis, derive the bilinear-space objects in that basis, identify their structural properties, and then establish basis-invariant criteria which implement these features. The end result is a set of Checks which can be performed in any basis, such that the model possesses a given symmetry group if and only if the potential passes these Checks.
The layout of the paper is as follows. In section II we outline the bilinear space technique, describe the products of the adjoint space vectors based on the SU (3) invariant tensors f ijk and d ijk , and then show the idea of dissecting the adjoint space with the aid of these vectors. These tools will play the crucial role in detecting symmetries in a basis-invariant way. Then, in following three sections, we apply these methods to all symmetry groups available in the 3HDM scalar sector, starting with the abelian ones, then continuing to non-abelian ones. We then conclude with an outlook of how to use the results of this paper in phenomenological scans of the 3HDM parameter space. Additional technical details and derivations are contained in Appendices.

A. Orbit space
We begin with a brief review of the bilinear formalism with specific application to 3HDMs [62,63].
We work with N = 3 Higgs doublets φ a , a = 1, 2, 3, all having the same electroweak quantum numbers. The most general renormalizable 3HDM potential can be compactly written as We construct the following 1 + 8 gauge-invariant bilinear combinations (r 0 , r i ): Here, t i = λ i /2 are generators of the SU (3) algebra satisfying with the SU (3) structure constants f ijk and the fully symmetric SU (3) invariant tensor d ijk . With the usual choice of basis for the Gell-Mann matrices λ i , these have the non-zero components as well as Group-theoretically, r 0 is an SU (3) singlet and r i realizes the adjoint representation of SU (3). The coefficient in the definition of r 0 is not fixed by this construction. We use here the definition borrowed from [62] but alternative normalization factors are possible [63]; the exact choice is not essential here.
In the Gell-Mann basis, the bilinears r i have the following form: The real vectors r obtained in this way do not fill the entire real eight-dimensional space R 8 (the adjoint space, whose vectors will be denoted as x), but a 7D manifold in it, which is called the orbit space. The points of this space are in one-to-one correspondence with gauge orbits within the Higgs fields space φ a . Algebraically, the orbit space is defined by the following (in)equalities [62]: A basis change in the space of Higgs doublets φ a → U ab φ b with U ∈ SU (3) leaves r 0 unchanged and induces an SO(8) rotation of the vector r i . However, not all SO(8) rotations can be obtained in this way; they must conserve, in addition, d ijk r i r j r k .

B. Constructions in the adjoint space
The main advantage of changing to the bilinear space is that the potential V becomes a quadratic rather than quartic function of variables: This generic expression holds for any NHDM. In 3HDMs, the potential (8) contains two 8D vectors M and L and the 8 × 8 real symmetric matrix Λ. The lack of the full SO(8) rotational freedom implies that it is not guaranteed that Λ can be diagonalized by a basis change. Nevertheless, Λ can always be expanded over its eigensystem, and eigenvalues and eigenvectors can be found, at least numerically.
We can now formulate the main idea which was recently proposed in [61] and which we fully develop in the present work.
These products respect group covariance: vectors F and D transform as adjoint SU (3) representations and, if needed, can be used in additional products. 1 These products were first used in [59] as building blocks of the basis-invariant algorithm to detect the usual CP symmetry in 3HDMs. For more than a decade, there were no follow-up studies. In fact, it was not broadly acknowledged by the community that these basis-invariant conditions for explicit CP conservation had been established in 3HDMs. Very recently, this approach was revived and further developed in [60] where the basis-invariant conditions for CP4 were established. These two papers provide the complete answer to the question of the basis-invariant recognition of a CP symmetry in 3HDMs and the same methodology enables the detection of other symmetries possible in 3HDMs. This is what we are going to achieve in the present paper.

C. Properties of the f and d-products
The vectors F and D defined in (9) obey certain remarkable properties, which follow from various relations among SU (3)-invariant tensors, see e.g. [64]. First, using the Jacobi identity d ijk f klm + d jlk f kim + d lik f kjm = 0, one observes that vectors F (ab) and D (ab) are always orthogonal: Any of these two vectors can be zero, but not simultaneously, because their norms satisfy For contraction of two d's, one has in SU (3) the following relation: Using it, one can derive This means that the non-linear action of d defined via a → D (aa) preserves the norm of unit vectors.

D. Detecting subspaces
The expressions for the tensors f ijk and d ijk make it clear that not all directions in the adjoint space R 8 are equivalent. There are basis-invariant features which distinguish various subspaces of R 8 with equal dimensions. We will see below that 3HDMs equipped with various symmetry groups differ by the subspaces in which the vectors M and L and the eigenvectors of Λ reside. Therefore, the first key step towards our goal is to develop a set of basis-invariant checks which detect that (eigen)vectors belong to a subspace of R 8 with certain properties.
The checks which are described in this section and elaborated in full detail in the appendix will be used to detect the direction x 8 , the subspace (x 3 , x 8 ), various patterns of the matrix Λ in its orthogonal complement among others. We stress that these checks detect certain basis-invariant conditions. It is never needed to actually switch to a preferred basis to perform a check.  (5), which are valid in any basis, one finds that D (aa) also stays in the In polar coordinates on the (x 3 , x 8 ) plane, this operation acts on the angular variable of a as α → π/2 − 2α. Hence, the three directions α = π/2, π/6, and 5π/6 are stable under this action (cf. [62] for more details on this construction). The first direction corresponds to a being aligned with then, in the appropriate basis, e (8) is along axis x 8 , and the matrix Λ takes the block-diagonal form with a 7 × 7 block and a stand-alone entry Λ 88 . Such an eigenvector does not have to be unique.
Next, let us find when two adjoint space vectors a and b can be simultaneously brought to the (x 3 , x 8 ) subspace. This is possible if and only if the corresponding traceless hermitian matrices A and B commute. Back in the adjoint space, this is equivalent to Thus, we obtain Check- (38): if Λ has two orthogonal eigenvectors a and b which satisfy (17), then there exists a basis change which brings both of them to the (x 3 , x 8 ) plane. The matrix Λ takes the block-diagonal form with a 2 × 2 block in this subspace and the 6 × 6 block in its orthogonal complement V 6 . Again, it is not guaranteed that such a pair of eigenvectors is unique.
This is only possible if D (aa) = −D (bb) . One can also show the converse: starting from D (aa) = −D (bb) for two orthogonal eigenvectors of Λ, one recovers Eq. (17).
Notice that passing Check- (38) does not guarantee that the two eigenvectors are aligned with the axes x 3 and x 8 . For that, one needs to require an extra condition, and the criterion for this to happen can be summarized as We thus formulate Check- (3)(8) then in an appropriate basis, e 8 is along needed, these eigenvectors can be aligned with the axes by a basis change. Thus, the matrix Λ takes in this basis the block-diagonal form with the diagonal entries Λ 11 , Λ 22 , Λ 33 , Λ 88 , and the 4 × 4 block in the orthogonal complement These simple examples give an overall impression of how one can detect subspaces in R 8 with distinct basis-invariant properties and ensure that Λ has certain block-diagonal form in an appropriate basis.
In the Appendix, we further develop this technique and derive several other checks. We also add here that, when deriving properties of certain subspaces, one often has a choice of which vectors to use, F or D. Most checks below we will make use of vectors D, although in some cases an equivalent formulation in terms of vectors F is also possible, in the light of the relations listed in the previous subsection.
Sparing the details presented in Appendices A and B, we give here a list of the checks for Λ, which detect various special subspaces or patterns inside subspaces.
For Λ matrices passing Check- (38), the 6D subspace V 6 can further split or can demonstrate special patterns.
For Λ matrices passing Check- (38) and Check- (12), the 4D subspace V 4 can still demonstrate special patterns characteristic for two non-equivalent implementations of U (1) symmetry: In the following sections, we will show how various symmetries groups imposed on the 3HDM scalar sector can be detected in the basis-invariant way via these Checks.

A. Rephasing symmetries
Let us first recapitulate the main features of the classification of abelian symmetry groups in the scalar sector of 3HDMs [30,31]. All abelian subgroups of SU (3), in a certain basis, can be represented by rephasing groups. 2 Only a few of them can be used to define models which do not possess additional accidental family symmetries. These groups are: All of them are subgroups of the maximal abelian group U (1) × U (1). Qualitatively, the larger the symmetry group is, the fewer are the free parameters remaining in the potential, and the tighter are the conditions one needs to impose to define the model.
The maximal abelian group U (1) × U (1) (maximal torus) is a two-parametric subgroup of SU (3) of the following transformations: Notice that the two transformations U (1) 1 and U (1) 2 differ by their eigenvalue multiplicities. There is no basis change which would map any U (1) 1 -transformation into any U (1) 2 transformation. Also, If one wants to construct the maximal torus in P SU (3) ≃ SU (3)/Z(SU (3)), one would get the same Let us now write the 3HDM potential symmetric under U (1) × U (1): It contains 3 quadratic terms and 9 quartic terms, all with real coefficients. The model is automatically CP -conserving; the CP symmetry can be generated, for instance, by the usual conjugation.
In the adjoint space, one gets scalars the two vectors and with One observes that Λ has a generic 2 × 2 block in the (x 3 , x 8 ) subspace, while in the subspace V 6 (Eq. (14)), it has the diagonal, pairwise-degenerate structure within the subspaces (x 1 , x 2 ), (x 4 , x 5 ), and (x 6 , x 7 ). The two vectors M and L have non-zero components in the (x 3 , x 8 ) subspace.
Using the results of sections II D and appendix B 1, we can easily formulate necessary and sufficient basis-invariant conditions for the 3HDM potential to be U (1) × U (1) symmetric: • the matrix Λ must pass Check- (38) and Check- (12)(45)(67); • each pair of eigenvectors in Check-(12)(45)(67) must correspond to the same eigenvalue; • the vectors M and L must be orthogonal to the six eigenvectors of V 6 .
Groups U (1) 1 and U (1) 2 in (22) are distinct, and imposing each of them constrains the potential in a different way. Imposing U (1) 1 leads, in addition to V 0 (Eq. (23)), to one more term: with complex λ 5 . Since λ 5 is the only complex parameter, one can rephase the doublets to make it real, which implies that U (1) 1 automatically leads to explicit CP conservation. In the adjoint space, the blocks of Λ in (x 3 , x 8 ) and (x 1 , x 2 ) are unchanged, while within the subspace V 4 (Eq. (20)), the block is modified by the additional term to This pattern in V 4 can be detected by conditions formulated in Appendix B 3. Thus, the necessary and sufficient basis-invariant conditions for the 3HDM potential to be U (1) 1 -symmetric are: • the matrix Λ passes Check- (38) and Check- (12); • the two eigenvectors of Check- (12) correspond to the same eigenvalue; • within V 4 , Λ passes Check-U (1) 1 ; • the vectors M and L are orthogonal to the six eigenvectors in V 6 .
In contrast to U (1) 1 , U (1) 2 allows for several new terms in addition to V 0 : All coefficients here can be complex. Even if one sets some of them real by a basis change, several complex coefficients will remain. Thus, the U (1) 2 -symmetric 3HDM can be explicitly CP violating.
In the adjoint space, one sees that vectors M and L can now have unconstrained components in the subspace (x 1 , x 2 , x 3 , x 8 ). The matrix Λ has a block-diagonal form with two blocks 4 × 4. The block in the subspace (x 1 , x 2 , x 3 , x 8 ) is generic, and therefore its eigenvalues are unconstrained. The block in its orthogonal complement V 4 shows the following pattern: which is different from (29). Thus, the necessary and sufficient basis-invariant conditions for the 3HDM potential to be U (1) 2 -symmetric are: • the matrix Λ passes Check-(4567) described in Appendix A 3; • within V 4 , Λ passes Check-U (1) 2 described in Appendix B 3; • the vectors M and L are orthogonal to the four eigenvectors in V 4 .
If one keeps, out of all terms in V U (1) 2 , onlyλ 5 (φ † 1 φ 2 ) 2 + h.c., then the potential is invariant not only under U (1) 2 but also under the Z 2 subgroup of U (1) 1 , which flips the sign of φ 1 . Since we are left with only one complex coefficient, this model is explicitly CP conserving.
The new term preserves the block-diagonal form of Λ in Eq. (26) apart from the 2 × 2 block in the (x 1 , x 2 ) subspace. This block becomes generic, so that the eigenvalue degeneracy is lifted. Thus, the basis-invariant conditions for the U (1) × Z 2 3HDM are the same as for U (1) × U (1) 3HDM with only this condition relaxed.
Restricting the previous case to the discrete subgroup of arbitrary sign flips, one obtains the famous Weinberg model with the symmetry group Z 2 × Z 2 [9]. The Higgs potential contains, in addition to V 0 , the following three terms: where all coefficients can be complex. If Im(λ 12λ23λ31 ) = 0, then it is impossible to make all coefficients real by any basis change, and the model is explicitly CP violating. 3 If it is real, then the model is explicitly CP conserving and is known as Branco's model [10,11].
In the adjoint space, the generic form within the subspace ( The necessary and sufficient basis-invariant conditions for the Z 2 × Z 2 -symmetric 3HDM are given by the simplified version of the U (1) × U (1) case: • the matrix Λ passes Check- (38) and Check-(12)(45)(67); • the vectors M and L are orthogonal to the six eigenvectors in V 6 .
Explicit CP conservation, that is, whether this is Weinberg's or Branco's model, can be detected by Check-(257) described in Appendix A 2 and first derived in [59].
The Z 4 -symmetric 3HDM can only arise as a particular case of the U (1) 1 3HDM. The Z 4symmetric potential contains, in addition to V 0 , two extra terms: Since there are only two complex coefficients, they can be made real via rephasing, and the model is explicitly CP conserving. The matrix Λ has the familiar features: a generic block in the subspace (x 3 , x 8 ), a generic block in the subspace (x 1 , x 2 ), and the block-diagonal structure (29) in V 4 .
The basis-invariant conditions are the same as for U (1) 1 , with the removal of the condition of the eigenvalue degeneracy within the subspace (x 1 , x 2 ), i.e.
The Z 3 -symmetric 3HDM can also only arise as a particular case of the U (1) 1 3HDM. Its potential contains, in addition to V 0 , three extra terms: where all coefficients can be complex. Even if one makes two of them real (for example λ 6 and λ 7 ), the other (e.g. λ 5 ) can still be complex, thus the possibility of explicit CP violation remains.
The matrix Λ still has a generic block in (x 3 , x 8 ), while within V 6 it takes the following form: Reλ 6 Imλ 6 Reλ 7 Imλ 7 0 λ ′ 12 Imλ 6 −Reλ 6 −Imλ 7 Reλ 7 Reλ 6 Imλ 6 λ ′ 13 0 Reλ 5 −Imλ 5 This matrix has three twice-degenerate eigenvalues. In appendix B 2, we prove that this pattern emerges if and only if all three pairs of eigenvectors corresponding to the same eigenvalue pass Check-Z 3 . Therefore, the basis-invariant necessary and sufficient conditions for Z 3 -symmetric 3HDM are: • the matrix Λ passes Check-(38); • the six eigenvalues of Λ within V 6 display 2+2+2 degeneracy, and each pair of the eigenvectors passes Check-Z 3 ; • the vectors M and L are orthogonal to the six eigenvectors in V 6 .
Explicit CP conservation within Z 3 3HDM implies that, in a certain basis, all coefficients are real.
The 6 × 6 block then splits into two 3 × 3 blocks, which are closed under the f -product, so that this feature can be detected by Check-(257).

H. Z 2 3HDM
Finally, the smallest symmetry group one can impose is Z 2 generated, for example, by the sign flip of doublet φ 3 . In the adjoint space, the only feature one observes is that Λ splits into two 4×4 blocks: one in the (x 1 , x 2 , x 3 , x 8 ) subspace and the other in V 4 . The structure of each block is unconstrained.
In Section A 3 we formulated Check-(1238) which detects exactly this splitting of Λ. It must be accompanied with the requirement that vectors M and L are orthogonal to the eigenvectors from V 4 .
In summary, in this section we gave basis-invariant conditions for each rephasing symmetry group in 3HDM, starting from the largest one U (1) × U (1) and then descending to its subgroups. As the symmetry is reduced, we see that qualitatively the conditions are gradually relaxed.

A. U (2)-symmetric 3HDM
We now move to the symmetry groups with two-dimensional irreducible representations. As before, we begin with the largest subgroup of SU (3) with 2D irreducible representation, U (2) ≃ SU (2) × U (1). In the basis where SU (2) transformations act non-trivially on φ 1 , φ 2 and U (1) transformations are of the type U (1) 2 , the potential takes the form which is the U (1) × U (1) potential (23) with the additional constraints and In the adjoint space, one sees that the vectors M and L, in this basis, are along axis x 8 . The only off-diagonal element of Λ in (26) is now zero, Λ 38 = 0, so that Λ becomes diagonal with the following unit blocks: and Λ 8 = 4(λ 1 + λ 3 − λ 13 )/3 − λ ′ 12 /3. The converse is also true: if L, M are parallel to x 8 and Λ exhibits this pattern, then the potential is invariant under U (2) symmetry.
To determine the basis-invariant conditions for the U (2) symmetry to be present, we first need to detect the special direction x 8 . This is done by Check-(8) described in section II D: if there exists an eigenvector e (8) satisfying (16), then in the appropriate basis it can be aligned with the positive direction of axis x 8 . We also require that L and M are aligned in the same direction. Next, one must observe that the eigenvalues of Λ display the degeneracy pattern 3 + 4 + 1, with the non-degenerate eigenvalue corresponding to e (8) . Moreover, the eigenvectors corresponding to the triple-degenerate eigenvalue must pass Check-(123) described in Appendix A 2. If all these conditions are satisfied, the model has the U (2) symmetry.

B. O(2)-symmetric 3HDM
When going from SU (2) × U (1) to smaller groups with 2D irreducible representations, one first notices that imposing SU (2) alone automatically leads to an accidental U (1), bringing one back to the previous case. Thus, we consider next the symmetry group O(2) ≃ SO(2) ⋊ Z 2 . When describing SO(2) transformations, it is convenient to work in the basis where they are given either by orthogonal rotations in the (φ 1 , φ 2 ) subspace or by rephasing transformations from U (1) 1 . In the former case, the extra Z 2 can be generated by a reflection with respect to any direction in this subspace: with c δ = cos δ and s δ = sin δ, angle δ being a free parameter, while in the latter case the generator can be the transformation b In the real O(2) basis, the most general potential compatible with this symmetry contains, in addition to Eq. (36), the following terms: In the adjoint space, the matrix Λ takes the following form: In the rephasing basis, one takes V 0 as in (23), applies the conditions (37), and adds the U (1) 1symmetric terms (28) without any constraint on λ 5 . The resulting matrix Λ acquires a slightly different form: In both cases one observes that the eigenvalue degeneracy pattern becomes 1+2+2+2+1, where the non-degenerate eigenvalues can only correspond to x 8 and an eigenvector in the subspace (x 1 , x 2 , x 3 ).
To detect the presence of this symmetry group in a basis invariant way, we first detect the eigenvector e (8) via Check- (8) and then the three eigenvectors in the subspace (x 1 , x 2 , x 3 ) via Check-(123)(8), described in section II D. Next, one checks that two among the three eigenvalues within ( are degenerate, which singles out the corresponding subspace V 2 . The exact choice depends on the basis choice; the two forms of Λ in (43) and (44) correspond to two such choices.
With these conditions, one knows that Λ has a separate 4×4 block in V 4 with two twice degenerate eigenvalues and one needs to establish its structure. Applying the methods described in Appendix B 3 to any of the above two forms of Λ, one can establish the following basis-invariant conditions. Take a pair of eigenvectors a and b corresponding to the same eigenvalue. Then they satisfy Thus, the basis-invariant algorithm for detecting an O(2) symmetry in 3HDM is: • verify that Λ passes Check- (8) and Check-(123)(8); • check that at least two of the eigenvectors from the subspace (x 1 , x 2 , x 3 ) correspond to the same eigenvalue; • check that the remaining four eigenvectors from V 4 also correspond to two twice degenerate eigenvalues, and the eigenvectors in each pair satisfy (45).
• check that L and M are aligned with e (8) .
If one starts with the Z 4 symmetric model given by V 0 in (23) and V Z 4 in (33) and imposes the conditions (37), then the potential acquires yet another symmetry of order 2 given by (41). No other conditions on parameters λ 5 andλ 12 are needed. The total family symmetry group is then D 4 ≃ Z 4 ⋊ Z 2 , on top of which one also has a CP symmetry. The basis-invariant algorithm for detecting this symmetry can be formulated as: • the matrix Λ passes Check-(3)(8) and Check-(12); • within V 4 , Λ passes Check-U (1) 1 ; • the vectors M and L are aligned with e (8) .

D. S 3 -symmetric 3HDM
To construct an S 3 -invariant 3HDM, one starts with the Z 3 -symmetric case with the potential V 0 in (23) and V Z 3 in (34), and imposes an additional symmetry b 2 (Eq. (41)). As before, one obtains the same constraints (37) as well as the new constraint on the Z 3 -symmetric parameters: Coefficients λ 5 , λ 6 , and λ 7 can still be complex with arbitrary phases, as for any phase choice for λ 6 and λ 7 , there exists a parameter δ in (41) such that b 2 is indeed a symmetry of the potential.
In the adjoint space, we see a picture similar to the previous case. The subspace (x 3 , x 8 ) splits into separate x 3 and x 8 subspaces, and the matrix Λ acquires two eigenvectors along these directions, e (3) and e (8) . The vectors L and M must be aligned with e (8) . The 6 × 6 block of Λ within the subspace V 6 keeps its form (35) but it is now constrained by the relation (46).
We find that the shortest way to implement it in the basis-invariant way is to calculate vectors and require them to be aligned with x 8 . Starting from (35), one finds that the only new conditions arise from their x 3 components: from which one immediately recovers (46).
In summary, the basis-invariant algorithm for S 3 -symmetric 3HDM is: • the matrix Λ passes Check-(3)(8); • the six eigenvalues of Λ within V 6 display the 2 + 2 + 2 degeneracy, and each pair of the eigenvectors passes Check-Z 3 ; • check that the four vectors L, M , K and K (2) are aligned with e (8) .
In general, the S 3 3HDM can be explicitly CP -violating. If one wishes to check if CP is explicitly conserved, one needs to perform the same Check-(257) which was discussed before.

E. Exotic CP situations
Finally, there are two situations in which one starts with abelian Higgs family symmetry groups but implements in addition a CP symmetry in such a way that the resulting symmetry group has 2D irreducible representation.
The first case is the 3HDM invariant under CP4. This model was proposed in [26] and the basis-invariant algorithm for detecting CP4 was presented in [60]. Formulated in the language of the present paper, this algorithm proceeds as follows, using the vectors in (47): • the matrix Λ passes Check- (8) and Check-(123)(8); • the four vectors L, M , K and K (2) are aligned with e (8) .
The second case is the unusual realization of CP symmetric Z 2 ×Z 2 model, when the CP symmetry is of order 2 but it does not commute with the Z 2 × Z 2 family symmetry group. Group-theoretically, the symmetry content is described by is a generalized CP symmetry which acts on Z 2 × Z 2 by transposing its generators a 1 and a 2 : (CP ) −1 a 1 CP = a 2 . This group can also be presented as generated by an order-4 CP transformation a 1 CP and the usual CP transformation, which do not commute. This model represents, therefore, a more constrained version of CP4 3HDM; we refer to [60] for a basis-invariant strategy of detecting it.

A. SU (3)-symmetric 3HDM
Moving to symmetry groups with irreducible triplet representations, we begin with the largest group available, SU (3). The SU (3)-symmetric 3HDM has only three terms in the scalar potential: The second line of Eq. (49) represents ( 8 i=1 r 2 i ) − r 2 0 , which is a non-positive quantity. Thus, in the adjoint space, this potential is characterized by vectors L = 0 and M = 0 and Λ = λ ′ 1 8 , which is invariant under all SO(8) rotations. Clearly, the potential will have this form in any basis, which will be immediately recognized. Still, we can formulate the basis-invariant condition for the SU (3) symmetry as absence of any vector and the full degeneracy among the eigenvalues of Λ. and V − = (x 2 , x 5 , x 7 ). Thus, the matrix Λ can now be written as Λ 1 1 5 + Λ 2 1 3 , with the two distinct eigenvalues Λ 1 and Λ 2 corresponding to V + and V − respectively.
The basis-invariant detection of the SO(3) symmetry consists in checking that vectors L and M are absent, detecting the 5 + 3 degeneracy pattern of the eigenvalues, and finally verifying that the eigenvectors corresponding to the triple degenerate eigenvalue satisfy Check-(257) described in Appendix A 2.

C. A 4 and S 4 -symmetric 3HDMs
Next, we pass to the discrete groups with irreducible triplet representation which can arise in the scalar sector of 3HDM. Two of them can be obtained as extensions of the Z 2 × Z 2 group by the permutation symmetries of three of its generators: A 4 ≃ (Z 2 × Z 2 ) ⋊ Z 3 and S 4 ≃ (Z 2 × Z 2 ) ⋊ S 3 . In the basis where Z 2 × Z 2 is given by the sign flips of individual doublets, the A 4 -symmetric potential is written as a constrained version of (23) and (32): Here, the parametersλ 12 ,λ 23 , andλ 31 can be complex with arbitrary phases but equal absolute values: If these conditions are satisfied, then the potential (50) possesses the A 4 -symmetry, in which the Z 3 generator is given by cyclic permutations of the doublets accompanied with suitable phase factors.
If, in addition, Im(λ 12λ23λ31 ) = 0, the symmetry group enlarges to S 4 . Indeed, one can switch to the basis where these three coefficients are real, and the potential becomes symmetric under any (not just cyclic) permutations of the three doublets. Notice that in either case, the model is explicitly CP -conserving.
In the adjoint space, one notices that L = 0 and M = 0, while the matrix Λ takes, just as in the Z 2 × Z 2 case, the block-diagonal form with blocks in the subspaces (x 3 , x 8 ), (x 1 , x 2 ), (x 4 , x 5 ), and (x 6 , x 7 ). However, the (x 3 , x 8 ) block is now simply 3λ 38 1 2 , while the other three 2 × 2 blocks within V 6 have identical pairs of eigenvalues λ ′ ± 2λ but arbitrarily oriented eigenvectors. For the S 4 -symmetric case, their orientation is correlated, though, and in a certain basis all eigenvectors in V 6 can be aligned with the axes, which renders the matrix Λ diagonal. In either case, one observes the eigenvalue degeneracy pattern 2 + 3 + 3. Notice also that by setting, in addition, 3λ 38 = λ ′ + 2λ, one would recover the SO(3)-symmetric case.
The basis-invariant algorithm for detection of the A 4 symmetry is: • verify that the matrix Λ passes Check- (38) with degenerate eigenvalues; • verify that Λ passes Check-(12)(45)(67) and displays three identical pairs of eigenvalues; • the vectors L and M are absent.
In order to detect the S 4 symmetry, one additionally requires that one of the triplets of V 6 eigenvectors sharing the same eigenvalue is closed under the action of d-product.
The symmetry group ∆(27) ⊂ SU (3) is generated by two order-3 transformations, which are traditionally chosen to be rephasing transformations diag(ω, ω 2 , 1) and cyclic permutations which can be accompanied by rephasings. 4 It turns out that ∆(27)-symmetric 3HDM automatically acquires an accidental Z 2 symmetry which makes the total symmetry group of the model ∆(54).
The general ∆(54)-symmetric 3HDM potential has the form similar to (50) but with the different last bracket: Just like in the Z 3 -symmetric case, the coefficients λ 5 , λ 6 , and λ 7 can be complex, but, in order for the potential to be invariant under cyclic permutations, they must have the same absolute values: One can perform rephasing transformations to set these three parameters equal toλ, whereλ 3 = λ 5 λ 6 λ 7 . Additionally, if these parameters satisfy Im(λ 5 λ 6 λ 7 ) = 0, then there exists a basis in which they all are real, up to powers of ω, and the model is explicitly CP -conserving.
In the adjoint space, one observes the absence of vectors M and L, and for Λ, the simple structure 3λ 38 1 2 in the (x 3 , x 8 ) block, and the residual 6 × 6 block in V 6 which has the same structure as (35) but with equal diagonal elements and with the off-diagonal elements satisfying the conditions (53).
The eigenvalues of this 6 × 6 block exhibit the 2 + 2 + 2 degeneracy pattern and are equal to In the CP -conserving case, two of the three real parts coincide, and the degeneracy pattern is promoted to 2 + 4.
The CP -conserving case corresponds to the situation where the four pairs of eigenvectors exhibit the above properties but the eigenvalue degeneracy pattern becomes 2 + 2 + 4.
Finally, the largest discrete symmetry group which can be imposed on the 3HDM scalar sector is Σ(36), which is twice larger than ∆(54). 5 It arises in the realλ basis if the coefficients of V ∆ (54) satisfy an additional constraint: 3λ 38 = λ ′ + 2λ. The potential then becomes symmetric under the following transformation of order 4: such that d 2 describes the transposition of φ 2 ↔ φ 3 . Adding d to the symmetry generators leads The basis-invariant path to this symmetry group is to observe the four pairs of eigenvectors satisfying the same conditions as for the ∆(54)-case, but with the eigenvalue degeneracy pattern 4 + 4.

VI. CONCLUSIONS AND OUTLOOK
In this paper, we solved the notoriously difficult problem of recognizing in a basis-independent way whether a 3HDM scalar potential has a symmetry. Similar methods for 2HDM existed for more than a decade, but generalizing them beyond two doublets proved challenging. Within 3HDM, prior to this work, it was known which symmetry groups G can be imposed on its scalar sector and how to write general potentials invariant under each G in a special basis, in which the generators of G take simple form. However it was always understood that if the same G-symmetric 3HDM was written in a different basis, the presence of G would be hidden and recognizing it would become very challenging.
Developing the ideas suggested very recently in [60] and [61], One can also investigate situations when a model is close to several symmetry situations simultaneously. In this case, one may observe and explore competing effects of proximity to the two symmetry groups. Such studies will generate not only numerical results but also a qualitative intuition of how one should build multi-Higgs-doublet models with desired phenomenological properties. We showed in section II D that the products of adjoint space vectors a and b can be used to identify basis-invariant features of the subspaces to which these vectors belong.
These products satisfy relations (13) and (11), which we now rewrite assuming vectors a and b are orthonormal: When applied to the eigenvectors of Λ, this technique can ensure that in an appropriate basis Λ has a block-diagonal form.

1D and 2D subspaces
The two examples given in the main text correspond to basis-invariant detection of 1D and 2D subspaces. Let us summarize them here for completeness.
• If a vector a satisfies D (aa) = −a, then there exists a basis in which a is aligned with +x 8 direction. When applied to the eigenvectors of Λ, this requirement constitutes Check- (8). No other basis-invariant condition detecting an 1D subspace with different properties exists. Check-(3)(8), respectively. Notice also that if it happens that two eigenvectors passing Check- (38) correspond to the same eigenvalue, one can always find their linear combinations which will pass Check-(3)(8).
As we already described in section II D, having identified an eigenvector e (8) via Check- (8), we can easily detect if there are three other eigenvectors spanning the subspace (x 1 , x 2 , x 3 ); this was formulated as Check-(123) (8). However, it is also possible to detect three eigenvectors from this 3D subspace even without the presence of e (8) .
Suppose one has three orthonormal vectors a, b, c which are closed under f -product: Then their respective hermitian matrices A, B, C form the su (2)  It is also possible that the three orthonormal vectors a, b, c, which are closed under f -product, need to be corrected by the factor 2: Then, the matrices A, B, C form the so(3) subalgebra of su(3). One can always rotate the three vectors to the subspace (x 2 , x 5 , x 7 ) or to other equivalent subspaces such as (x 2 , x 4 , x 6 ), etc. This property is the basis of Check-(257), which was used in [59] to detects explicit CP conservation in 3HDM.
The proof goes as follows. Denote a = (a 1 , a 2 , a 4 , a 5 , a 6 , a 7 ) and b = (b 1 , b 2 , b 4 , b 5 , b 6 , b 7 ) and compute the D-products explicitly. First, write down D = 0: Together with the orthogonality condition a b = 0, they lead to This implies the following structure for b: with some real coefficients σ, σ ′ , σ ′′ .
Next, from D (aa) = D (bb) within the subspace (x 3 , x 8 ) as well as from the normalization condition a 2 = b 2 = 1, we see that σ's can only be ±1.

Z 3 pattern inside V 6
Let us relax the conditions (B1) which defined Check- (12) and require now that which we call Check-Z 3 . That is, we now allow for non-zero vectors D (ab) and D (aa) − D (bb) provided they belong to V 6 . Repeating the calculations of section B 1, we see that all components of a and b can be non-zero. However b must still be of the form (B4) with σ = σ ′ = σ ′′ = ±1.
Next, suppose the two eigenvectors of Λ, which we denote e and e ′ , satisfy (B7) and correspond to the same eigenvalue λ. It can be immediately checked that their contribution to the eigensystem expansion for Λ, e i e j + e ′ i e ′ j , has the following form: It is remarkable that this block has exactly the same form as in the Z 3 -symmetric 3HDM, Eq. (35).
Therefore, if the eigenvalues of Λ within V 6 are pairwise degenerate, and if the three corresponding pairs of eigenvectors satisfy Check-Z 3 given in Eq. (B7), then we obtain the Z 3 -symmetric model.
Notice that the three pairs of eigenvectors may be in arbitrary orientation with respect to each other; apart from mutual orthogonality, there are no constraints.
Its square and cube have the same form with squared and cubed eigenvalues, respectively. This can happen only if each eigensystem e i e j + e ′ i e ′ j has the form (B8). Contracting it with √ 3d ijk gives the vector D (ee) + D (e ′ e ′ ) , and one can verify by explicit calculation that it indeed belongs to (x 3 , x 8 ).
Next, we checked with Mathematica that each pair of eigenvectors (e, e ′ ) of this matrix has the form of vectors a and b as in (B4). That is, not only are the eigenvectors (e, e ′ ) themselves orthogonal and equally normalized but so are their 2D components within the subspaces (x 1 , x 2 ), (x 4 , x 5 ) and (x 6 , x 7 ). This immediately implies that D (ab) and D (ee) − D (e ′ e ′ ) cannot have any components in the (x 3 , x 8 ). Thus, we arrive at all three conditions of Check-Z 3 in (B7).
Notice that, unlike the previously considered example, this set of conditions explicitly distinguishes subspaces (x 1 , x 2 ) and (x 3 , x 8 ). Thus, it can be used only after we have already passed Check- (38) and Check- (12). Now, once again, suppose that within V 4 , matrix Λ has two degenerate eigenvalues each corresponding to a pair of eigenvectors which satisfy (B12). Then we say Λ passes Check-U (1) 1 . The 4 × 4 block constructed via the eigensystem expansion now has the following form: