Theory of inversion-Z4 protected topological chiral hinge states and its applications to layered antiferromagnets

We study positions of chiral hinge states in higher-order topological insulators (HOTIs) with inversion symmetry. First, we exhaust all possible configurations of the hinge states in the HOTIs in all type-I magnetic space groups with inversion symmetry by studying dependence of the sign of the surface Dirac mass on surface orientations. In particular, in the presence of glide symmetry, for particular surface orientations, the surface Dirac mass changes sign by changing the surface terminations. By applying this result to a layered antiferromagnet (AFM), we find a difference in the hinge states between the cases with an even and odd number of layers. In the case of an even number of layers, which does not preserve inversion symmetry, positions of hinge states are not inversion symmetric. Nonetheless, these inversion-asymmetric hinge states result from the bulk topology. We show that their inversion-asymmetric configurations are uniquely determined from the symmetries and the topological invariant.


I. INTRODUCTION
The discovery of topological insulators (TIs) have triggered intensive studies on topological aspects in electronic structures of solids 1,2 . Three-dimensional (3D) TIs have gapless surface states which are protected by timereversal (T ) symmetry. On the other hand, these surface states can be gapped by breaking T symmetry while preserving topological properties of the bulk; such TIs are called magnetic TIs. Among magnetic TIs, those characterized by the quantized value of the Chern-Simons axion angle θ = π 3-13 are known as axion insulators (AXIs). The simplest AXIs have been proposed in inversion (I) symmetric TIs in an external magnetic field and in a TI doped with magnetic atoms without breaking I symmetry. I symmetry pins the axion angle θ to π even when T symmetry is broken. AXIs have the quantized magnetoelectric effect from the nontrivial axion angle θ = π 4-6 , and the surfaces of AXIs have a half quantum Hall effect 4,9 . Recently, EuIn 2 As 2 11 and MnBi 2 Te 4 14 have been proposed as layered antiferromagnetic TIs, and then they have been studied extensively [15][16][17][18][19][20][21][22][23][24][25][26] . In MnBi 2 Te 4 , the combined symmetry S = T τ 1/2 leads to a Z 2 topological classification in the absence of T symmetry 27 , where T and τ 1/2 represent a T operator and a half-translation one respectively.
Higher-order TIs (HOTIs) have been proposed as a new class of TIs  . 3D HOTIs have topological onedimensional states along hinges of the systems, which are called hinge states. Hinge states arise from nontrivial higher-order topology of the bulk. Among various classes of HOTIs, one class of HOTIs is protected by I symmetry 10,11,39,40,[43][44][45][46] . The HOTIs without T symmetry have chiral hinge states (CHSs) 10,11,39,40,43 , whereas the HOTIs with T symmetry have helical hinge states [44][45][46]71 . In the magnetic systems only with I symmetry, the topological phases are characterized by three weak Z 2 indices and the strong Z 4 index µ 1 72-77 , and CHSs always appear when all the weak indices vanish and µ 1 = 2 78,79 . Therefore, magnetic TIs with I symmetry and µ 1 = 2 have CHSs, and remarkably, they are AXIs with θ = π 9-11, 13 . The positions of the CHSs found are always I symmetric 10,11,39,40,43 .
In this paper, we find all the possible patterns of positions of CHSs in HOTIs from I symmetry. We exhaust all the patterns by studying dependence of the sign of the surface Dirac mass on surface orientations for each space group. In particular, in the presence of glide symmetry, for particular surface orientations, the surface Dirac mass changes sign by changing the surface terminations. By applying this result to a layered antiferromagnet (AFM), we find a difference in the hinge states between the cases with an even and odd number of layers, and we find emergence of I-asymmetric hinge states (IAHS) in the case of an even number of layers. Moreover, we show that IAHS result from the bulk Z 4 topology protected by I symmetry, and they generally appear in antiferromagnets (AFMs) with an even number of layers and the non-trivial Z 4 index. In addition, we show that their Iasymmetric positions are uniquely determined from an interplay of the symmetries and topology, which is also discussed in the axion insulator EuIn 2 As 2 11,23,24 .
The organization of the paper is as follows. In Sec. II, we find all the possible patterns of positions of CHSs in HOTIs in all type-I MSGs with I symmetry. In Sec. III, we discuss CHSs in layered antiferromagnetic HOTIs, and then we show that the positions of the CHSs depend on the parity of the number of layers N . In Sec. IV, we perform calculations on the tight-binding model to confirm our theory in Sec. III. Our conclusion is given in Sec. V.

II. TYPE-I MAGNETIC SPACE GROUPS WITH INVERSION SYMMETRY
First, we consider positions of the CHSs in all the 92 space groups (SGs) with I symmetry or, equivalently the 92 type-I magnetic space groups (MSGs) with I symmetry. To this end, we consider a Dirac Hamiltonian on a surface of the system. In the simplest case, the surface Dirac Hamiltonian is written as where σ = (σ x , σ y , σ z ) are Pauli matrices,k = (k 1 , k 2 ) is a wave-vector along the surface, and λ and m are real constants. On every surface, we define the mass m for the surface Dirac Hamiltonian, whose general form is discussed in Appendix A 1. We first assume that the surface Dirac mass m n is determined by the surface orientation, where n is the normal vector of the surface. We show the dependence of m n on n in a system with a shape of a sphere, with a point on the sphere identified as a surface normal vector n (Fig. 1). In the I symmetric HOTIs without T symmetry, the sign of the mass term is reversed by I operation, i.e. m −n = −m n 43 . Thus, the domain wall with m n = 0 appears in a I-symmetric manner, and on this m n = 0 line, the CHS appears in a clockwise direction around the region with positive m n (see Fig. 1(a-1)). Therefore, in the rod geometry along the z direction, the CHSs follow from the distribution of m n on the equator of the sphere ( Fig. 1(a-2)).
Next, we consider the cases with n-fold rotational (C nz ) symmetry around the z-axis in addition to I symmetry. In the presence of I and C 3z symmetries, i.e. the point group 3, the CHS is invariant under C 3z symmetry ( Fig. 1(b)). In the cases with C 2z symmetry, the CHS on the sphere is always a great circle on the mirror xy plane due to mirror symmetry M z (Fig. 1(c)). In this case, chiral surface states protected by mirror symmetry appear on the side surfaces in the rod geometry along the z-axis. Meanwhile, if all the surfaces of the crystal are not mirror symmetric, CHSs appear. Furthermore, in the cases with C 4z or C 6z symmetries, the distributions of the mass terms are the same as those with C 2z symmetry. Thus, the CHS on the sphere is along a great circle in the point groups 2/m, 4/m and 6/m. Similarly, in other point groups, the patterns of CHSs are classified into the five patterns ( Fig. 1(d-h)), where the CHSs are located on great circles on the mirror planes. As shown in Appendix D, the sum of the mirror Chern numbers on k z = 0 and k z = π planes, is a non-zero odd number when µ 1 = 2. Thus, the system with mirror symmetry has gapless surface modes protected by mirror symmetry when µ 1 = 2, which makes the mass to vanish on the intersection between the sphere and the mirror plane.
Notably, while on a mirror-invariant surface, the surface states are helical due to the nontrivial mirror Chern number, the hinge states predicted here are chiral as shown in Fig. 1(c-h). It is not a contradiction. For example, let us consider a 3D system of an I-symmetric alternate stacking of Chern insulator layers with Chern numbers ±1 along the z-direction, with each layer being M z invariant. Even with M z -invariant interlayer coupling, there appear gapless helical surface states on the (100) surface due to a nontrivial mirror Chern number. On the other hand, if we cut out a finite crystal with I symmetry, the crystal consists of N + 1 layers with the Chern number +1 and N layers with the Chern number −1 or vice versa to preserve I symmetry. Thus, on the (100) surface, we should consider the N → ∞ limit, and the surface states are helical. On the other hand, for an I-symmetric crystal with a finite N having no mirrorsymmetric surfaces, by moving states of opposite chirality towards each other without breaking I symmetry, the chiral edge states hybridize to result in a non-vanishing Chern number (N + 1) · 1 + N · (−1) = 1, leading to CHSs. Thus, the massless line on the mirror planes in Fig. 1(c-h) corresponds to both helical surface states and CHSs.
This approach of the mass term is used in the previous works 74,80 to classify the topological phases of topological crystalline insulators with T symmetry. Therefore, these previous works are not intended to list possible positions of hinge states. In contrast, in this paper, we focus on the position of CHSs in the system without T symmetry, not on the classification of topological phases. As we discuss in Appendix A 1, in the system without T symmetry, the transformation property of the mass term under rotation operations is different from the case with T symmetry, which gives crucial difference in the positions of CHSs. Next, we consider the cases with glide symmetry. In systems with glide symmetry, the glide-Z 2 topological invariant is defined 81,82 . Here, the question is a relationship between the glide-Z 2 invariant and the inversion-Z 4 invariant. To see this, we consider minimal SGs which contain I and glide symmetries. These are three such SGs, No. 13, 14 and 15. The SGs No. 13 and 14 are for the simple (primitive) lattice, with the inversion center inside the glide-mirror plane in SG No. 13 and outside it in SG No. 14. The SG No. 15 is for the base-centered lattice. It is found that in SG No. 13 and 15, a half of the inversion-Z 4 invariant µ 1 is equal to the glide-Z 2 invariant 83,84 , while in SG No. 14, it is equal to a sum of the glide-Z 2 invariant and a half of the Chern number, where the Chern number is even 83 . In either case, because we are considering cases with a vanishing Chern number, a nontrivial inversion-Z 4 invariant µ 1 = 2 automatically means a nontrivial glide-Z 2 invariant. Thus in our case of HOTIs, on the surfaces preserving the glide symmetry, there should appear topological gapless states, and the mass is zero.
To study the dependence of the surface mass on the surface orientation, we consider a system with a glide symmetry {M y |00 1 2 }. In this case, on the glide-invariant surfaces, which are perpendicular to the glide-mirror plane and along the direction of a fractional translation of the glide operations, the gap is zero in the HOTIs because of the nontrivial glide-Z 2 invariant ( Fig. 2(a)). More generally, by combining integer translations in the x, y and z directions with this glide operation {M y |00 1 2 }, it follows that {M y |a, b, c + 1 2 } is also among the symmetry operations of the system, where a, b, c are integers. In this case, the Miller index of the corresponding glideinvariant surface is (2c + 1, 0, −2a), and the surface Dirac mass on this surface is zero. Therefore, on the (α, β, γ) surface with β = 0, α ≡ 1 (mod 2), and γ ≡ 0 (mod 2), the Dirac mass is zero.
Here, we note the following point: so far we assume that the sign of m n is uniquely determined by the surface orientation, but the assumption is violated when the system has glide symmetries. For example, we consider a glide symmetry {M y |00 1 2 } once again. Suppose the surface mass term m for the (001) surface z = 0 is positive. Then from the glide symmetry, the (001) surface z = 1/2 has a negative mass. Then the glide symmetry requires two terminations for the (001) surfaces to have opposite mass signs, violating our assumption. Other than the (001) surface, there are surfaces with such ambiguity of the mass sign. In general, the (α, 0, γ) surfaces have such ambiguity if γ is an odd number, where α and γ are integers, with its details in Appendix A 4. Thus, there are an infinite number of surface orientations on which the mass ambiguity arises. To summarize, when the surface normal is within the glide-mirror plane on the sphere shown in Fig. 2(b), two cases for the surface mass coexist: the mass on the (α, 0, γ) surface is zero when α ≡ 1 (mod 2) and γ ≡ 0 (mod 2), while the mass is indeterminate when γ ≡ 1 (mod 2). Thus, the mass with the surface normal n within the glide mirror-plane sensitively depends on the surface orientation, and it is no longer a smooth function of n. In the mass distribution on the sphere, such n forms a great circle on the glide-mirror plane. We show  Fig. 1 (a-1, b−h) and the 6 types with glide symmetries (Types 3 -8 ) shown in Fig. 2(b-g). PG denotes the name of the point group. In Types 1-8, the MSGs do not include glide symmetries. On the other hand, in Types 3 -8 , the MSGs include glide symmetries, and these cases correspond to the mass distributions which are obtained by replacing the mirror planes in Types 3-8 with glide-mirror planes.

PG
Type-I MSG Type of CHS this great circle as a broken circle, in contrast with the solid circle on the mirror plane, where the mass is zero. We call the cases where the mirror planes in Types 3-8 are replaced with a glide mirror plane as Types 3 -8 (Figs. 2(b-g)).
Next, we consider the cases with screw symmetries. The combination between I symmetry and screw {C 2z |00 1 2 } symmetry leads to the mirror symmetry {M z |00 1 2 }, and the mirror Chern number is nontrivial when the Chern number on k z = 0 plane is zero and µ 1 = 2 as discussed in Appendix E. It leads to the emergence of the surface states protected by mirror symmetry (Type 3 in Fig. 1).
In this way, we identified all the 14 possible patterns of the mass distribution on the sphere and the resulting positions of CHSs (Types 1-8 in Fig. 1(a-h) and Types 3 -8 with glide symmetries in Fig. 2(b-g)) for all the 92 type-I MSGs with I symmetry as shown in Table I. We obtain this result using an observation that m n behaves as a pseudoscalar with its details in Appendix A. Here, we note the meaning of the spherical figures in Figs. 1 and 2. They represent dependence of the surface Dirac mass on the surface normal n. Therefore, for a given crystal shape with various facets, we apply one of the 14 patterns in Figs

III. CHIRAL HINGE STATES IN ANTIFERROMAGNETIC HIGHER-ORDER TOPOLOGICAL INSULATORS
Based on the mass distribution m n dependent on the surface normal vector n, one can determine positions of CHSs for a given crystal shape. In Sec. II, we find an interesting possibility in HOTIs with glide symmetry. Namely, for particular choices of surface orientations, the surface Dirac mass changes its sign, when the surface termination is shifted by a half of the lattice constant. Since the mass distribution as a function of n determines the positions of CHSs, our finding in Sec. II suggests that in such cases, a change in the surface termination, while keeping the surface orientation specified by n, will change the positions of the CHSs. To show this, we consider an example of a glide-symmetric system in cylindrical geometries along the direction of the fractional translation of glide symmetry, and we choose the Types 3 and 5 in Fig. 2 as an example.
Here, we consider crystals in Type 3 in a shape of a parallelepiped and in Type 5 in a shape of a hexagonal prism ( Fig. 3(a)). I-symmetric 3D layered antiferromagnetic HOTIs with µ 1 = 2. Here, the staggered magnetization is along the z-axis, which is along the stacking direction. In this section, we show that the positions of the CHSs depend on the parity of the number of layers N , and IAHS generally appear in the cases with even N , where I symmetry is broken. We assume that each layer has I symmetry. First, we consider CHSs along the direction of the stacking, i.e. the z direction as shown in Fig. 3(a) which follow from Fig. 1. Here we assume that the number of layers is much larger than the penetration depth of the CHSs. In this case, CHSs along the stacking direction always emerge irrespective of the number of layers because of the topological property in the bulk, i.e. µ 1 = 2 78 .
Next, we consider behaviors of CHSs at the corners A and B in Fig. 3(b). At each corner, the number of incoming hinge modes should be equal to that of outgoing hinge modes because otherwise a charge will be accumulated at a corner in proportion with time, due to the fact that each hinge mode provides one-channel transport. This argument is similar to the one in Ref. [85] applied for chiral edge currents in a two-dimensional (2D) insulating ferromagnet. Therefore, CHSs should appear either along (i) or along (ii) between the corners A and B (Fig. 3(b)). In addition, a similar discussion is applicable to the corners

A. Cases with an odd number of layers
First of all, we consider the cases with odd N with the magnetization of each layer given by ↑↓ · · · ↓↑, which represents the magnetizations of the individual layers from top to bottom. When N is odd, the system has I symmetry, and such an I-symmetric 2D slab of a 3D HOTI with I symmetry is shown to be a 2D Chern insulator with the Chern number C z slab ≡ 1 (mod 2) 10,39,79 . Here the Chern number C z slab in 2D systems is defined by Here is the Berry curvature written in terms of the Berry connection of the occupied bands, (A µ (k)) αβ = i u α | ∂ µ |u β . In the present case of the layered antiferromagnet, when N = odd, I symmetry is preserved, and the slab Chern number C z slab (mod 2) since we are considering HOTIs with µ 1 ≡ 2 (mod 4). In particular, for odd N with ↑↓ · · · ↓↑ and ↓↑ · · · ↓, we can choose C z slab = 1 and C z slab = −1 respectively without losing generality (see Appendix C), leading to the positions of CHSs shown in Figs. 3(c) and 3(d).

B. Cases with an even number of layers
Next, when N is even, I symmetry is broken and C z slab ≡ 1 (mod 2) does not hold. For even N with ↑↓↑↓ · · · ↓, it can be understood as ↑↓ · · · ↑ + ↓↑ · · · ↓, i.e. the composition of odd-N layers with C z slab = 1 and odd-N layers with C z slab = −1. The resulting positions of CHSs are shown in Fig. 3(e) because two counterpropagating CHSs on the same hinge will hybridize and open a gap. Likewise, for even N with ↓↑ · · · ↑, the CHSs are as shown in Fig Interestingly, though a slab with even N does not preserve I symmetry, it has IAHS due to bulk topology protected by bulk I symmetry. It is also interesting that the positions of the IAHS for even N are also uniquely determined in this case, and they are different from those for odd N . EuIn 2 As 2 with even N possesses IT symmetry whereT ≡ C 2z T , from which the Chern number is when N is even as proved in Appendix B.

IV. MODEL CALCULATIONS
In this section, we use a tight-binding model of a layered AFM showing a HOTI phase to study the behaviors of the IAHS. This model was proposed as a model of a HOTI with CHSs in Ref. [40]. This model is an alternate stacking of layers of the Haldane model within the xy plane with the Chern number = ±1 86 , and their magnetizations alternate correspondingly (Figs. 4(a) and 4(b)). The Haldane model is a tight-binding model on the honeycomb lattice representing a ferromagnet, and its Hamiltonian within the α-th layers is written as where i and j run over the sites in the layer α, t 1 is the hopping strength for the first-neighbor pairs ij , t 2 is that for the second-neighbor pairs, and ν ij = 1 (ν ij = −1) if the second-neighbor hopping path is counterclockwise (clockwise) in the hexagonal plaquette. The Fermi energy is set to be E F = 0. We assume t 1 and t 2 to be negative; then each layer is a Chern insulator with the Chern number C = (−1) α+1 . Therefore, the Chern numbers of the individual layers change alternately, and so does their magnetization. Therefore, the model is a layered AFM. Next we introduce a hopping t z between the layers as shown in Fig. 4(b), and this Hamiltonian is given by The overall Hamiltonian is H = H xy + H z , and it has I symmetry in the bulk. This interlayer coupling breaks S(= T τ 1 2 ) symmetry but preserves bulk inversion symmetry, which leads to gapped surface states and gapless CHSs. Here, we assume that the interlayer coupling is weak so that the gap remains open even when t z is continuously changed to zero. Then the topological properties of the layered system are the same as those of a stack of decoupled 2D Chern insulators. Therefore, C z slab is obtained by the sum of the Chern number of each layer.
In 3D systems, there are eight time-reversal invariant momenta (TRIM) denoted by Γ j . The topological phases are characterized by the Z 4 index for I symmetric systems in class A, one of Altland-Zirnbauer symmetry classes 88 . The Z 4 index is defined as 73 where n ± (Γ j ) are the number of the occupied states with even-and odd-parity eigenvalues at the TRIM Γ j , respectively. The eight TRIM are shown in Fig. 4(c), and the parity eigenvalues at each TRIM are shown in Fig. 4(d). From these parity eigenvalues, the Z 4 topological index is µ 1 = 2 in this model, which leads to HOTIs with CHSs 78 .
In the following, we perform calculations for the tightbinding model using the PythTB package 87 . We calculate the band structure with the rod geometry along the z direction, and the result is shown in Fig. 5(a). This model has C 3z symmetry, and therefore the CHSs appear at the positions which are related by C 3z symmetry as shown in Figs. 5(a-1) and 5(a-2). Next we calculate the band structure with the rod geometry which is finite along the directions of z axis and of the primitive translation vector a 2 , and infinite along the a 1 direction. The results are shown in Figs. 5(b) and 5(c), for N = odd and N = even respectively. When N = odd ( Fig. 5(b-2)), the system is I symmetric, and then the positions of CHSs are I symmetric, in agreement with Fig. 3(c) with C z slab = 1. When N = even, the system is not I symmetric, and the positions of CHSs are as shown in Fig. 5(c-2), that is, IAHSs appear with C z slab = 0. The MSG of the tight-binding model is P 6 3 /m m c (#194.268). This MSG contains two kinds of symmetry operations: a glide symmetry {M y |00 1 2 }, and a combination of screw and T operations, T {C 2z |00 1 2 }, both of which lead to a sign inversion of the surface Dirac mass on the (001) surface. The MSG P 6 3 /m m c (#194.268) has a maximum subgroup type-I MSG No. 167. According to Table I, it is similar to the point group 3m with three glide symmetries (Type 5 in Fig. 2).
According to Ref. [11], EuIn 2 As 2 is a layered antiferromagnetic AXI with the magnetic moment along z axis ( Fig. 6(a-1)). The MSG of EuIn 2 As 2 is P 6 3 /m m c (#194.268), the same with our model. Thus, EuIn 2 As 2 is an ideal material to study the emergence of IAHS. Remarkably, the mass term in EuIn 2 As 2 will change its sign under I, and invariant under C 3z , which makes the mass term having an alternate sign between adjacent side surfaces of the hexagonal crystal shown in Fig. 6(a-2) and Fig. 3(c). Moreover, if N is even, which breaks I symmetry ( Fig. 6(b-1)), CHSs will exist in an I-asymmetric configuration (Figs. 6(b-2) and 3(e)).
In the AXIs, such as EuIn 2 As 2 , a quantized magnetoelectric effect is expected. For its observation, we need to attach electrodes so that they are electrically isolated from each other. The CHSs can short-circuit them, then obstructing the measurement. Our analysis shows that the case with N = even may facilitate the measurement since the IAHS consist of multiple loops.

V. CONCLUSION
In summary, we found all the possible configurations of CHSs in HOTIs in type-I MSGs with I symmetry through an analysis of the sign of the surface Dirac mass. The configurations are uniquely determined from each MSG as a massless line of the surface Dirac mass. Through this study, we found that in systems with glide symmetry, two surface termination with the same particular surface orientations have the opposite signs of the mass. This possibility leads to a drastic difference in positions of CHS in AXIs realized in layered AFMs between the cases with an even number of layers and an odd number of layers. In particular, in the case with an even number of layers, IAHS appear at the I-asymmetric positions because I symmetry is broken in the whole system. Nonetheless, IAHS are protected by I symmetry in the bulk, and they are characterized by the topological Z 4 invariant. This difference in the positions of CHSs can be observed by transport measurements through hinges. Furthermore, we predict that IAHS appear in an AXI EuIn 2 As 2 with an even number of layers.  term is given by where σ i (i = x, y, z) represent the Pauli matrices, (k 1 , k 2 ) is the wave-vector along the surface, and λ and m are real constants. Here, m represents the surface Dirac mass. In fact, this form of the Hamiltonian is just an example, and in general cases, the Hamiltonian H(k) on the surface with a unit normal vector n can be represented by where k 3 ≡ m is a Dirac mass. In addition, A ij are real, and we can always assume det(A) > 0, because otherwise we change the definition of m by m ↔ −m so that det(A) becomes positive. This process uniquely determines the sign of the mass. Furthermore, for the moment we assume that the sign of the mass term is determined uniquely by the surface normal vector n. Now we discuss a crucial difference between HOTIs with and without T symmetry. In HOTIs without T symmetry, the sign of the mass can be defined without ambiguity, because it is associated with the chiral direction of the CHS, i.e. the CHS goes around the region with a positive mass in the clockwise way. In contrast, for helical hinge states in HOTIs with T symmetry, one cannot define their flow direction (because they contain flows in both directions), and one cannot define the sign of the mass m n in a gauge-independent way.
In the following, we use the fact that the mass term in HOTIs with I symmetry is a pseudoscalar under pointgroup operations, namely, it changes sign under improper rotations such as I and mirror operations. This fact is justified from the following three properties of CHSs.
(i) It is established that the mass term in HOTIs with µ 1 = 2 changes sign under I operation, and therefore the mass term satisfies m −n = −m n . 43 Therefore, the domain wall with m n = 0 appears between the regions with a positive m n and that with a negative m n (see Figs. 7(a) and 7(b)). Along this massless line, CHS exists in a clockwise way around the m n > 0 region. In this way, CHSs appear along the domain wall with m n = 0. On the other hand, in normal insulators (µ 1 = 0), the sign of the mass term does not change under I operation.
(ii) Under the rotations, the sign of the mass term does not flip. One can see that it leads to a contradiction if the sign of the mass changes under rotation operations. For example, in a system with C 2z symmetry, if the sign of the mass term changes under C 2z operation, the CHS appears along the domain wall passing through the north pole and south pole, which are C 2z invariant points on the sphere. It obviously contradicts C 2z symmetry because of chiral nature. Thus, the mass does not change under C 2z , and therefore the CHS does not follow only from the C 2z symmetry. It is consistent with the trivial strong index of symmetry-based indicator 75 in type-I MSG No.3. In this argument, the chiral nature of the hinge states is crucial.
(iii) The sign of the mass term flips under a mirror operation. From this, the gapless modes appear on the mirror plane, which are identified as topological surface states on mirror-symmetric surfaces due to the nontrivial mirror Chern number, as we see in Appendix D.
Thus, we conclude that the mass term is a pseudoscalar in HOTIs with µ 1 = 2. In contrast, it is straightforward to see that in normal insulators with µ 1 = 0, the mass is a scalar, but not a pseudoscalar. Next, we consider cases with combinations between I symmetry and mirror symmetry. For example, we consider a system with I symmetry and M z symmetry with respect to xy plane, leading to the concomitant C 2z symmetry. In this case, the chiral edge mode is along a great circle on the sphere along the mirror plane as shown in Fig. 7(c). It leads to the gapless surface states on the side surfaces in the rod geometry along the z direction. These gapless surface states are protected by mirror symmetry. Meanwhile if the surfaces of the crystal is not mirror symmetric, CHSs appear. In the case with M x , M y and M z symmetries, the gapless surface states appear in the rod geometries along x, y and z directions (Fig. 7(d)). These gapless surface states are characterized by the non-trivial mirror Chern number. Here, the mirror Chern number for the M z symmetry is defined as where k z is 0 or π, corresponding to mirror-invariant planes. In Eq. A3 in spinful systems, C ± (k z ) are the Chern numbers in the mirror subspace M z = ±i defined as where F ± xy (k) is the non-Abelian Berry curvature in the ±i mirror subspace. In spinless systems, the mirror subspaces are defined by M z = ±1 instead. As shown in Appendix D, the sum of the mirror Chern numbers in both k z = 0 and k z = π sectors, is a non-zero odd number when µ 1 = 2. Thus, the system with mirror symmetry has gapless surface modes protected by mirror symmetry when µ 1 = 2.

Point groups
As discussed in the above, the mass term on a sphere includes various information of positions of CHSs. In this subsection, we consider the distributions of the mass term on spheres with various symmetries and classify them in terms of point groups. As discussed above, the CHS forms a loop protected by I symmetry in the point group 1 without other symmetries (Type 1, Fig. 1(a-1)). Here, we add rotation C nz (n = 2, 3, 4, 6) symmetries around z axis. The mass signs do not flip under rotations. In the case with C 3z symmetry, that is, in the point group 3, the CHS on the sphere is invariant under C 3z symmetry and I symmetry as shown in Fig. 1(b) (Type 2). On the other hand, in the case with C 2z symmetry, the combination of I symmetry and C 2z symmetry leads to the mirror M z symmetry with respect to xy plane, and the gapless line is along the great circle on the mirror plane ( Fig. 1(c)). The same applies to the case with C 4z and C 6z symmetries and we call this case Type 3.
In the point groups mmm and m3, CHSs appear along the three great circles because of M x , M y and M z mirror symmetries with respect to yz, zx and xy planes, respectively (Type 4, see Fig. 1(d)). In addition, CHSs appear along three great circles which are related by C 3z symmetry in the point group 3m as shown in Fig. 1(e) (Type 5). The point group 4/mmm has two mirror symmetries M xy and M xy that leave the x = y and the x = −y planes invariant, respectively, in addition to the mirror symmetries M x , M y and M z . Because of these mirror symmetries, CHSs appear along the five great circles as shown in Fig. 1(f) (Type 6). In the point groups 6/mmm and m3m, seven and nine great circles of gapless lines appear, respectively, because of additional mirror and rotation symmetries (Types 7 and 8, shown in Figs. 1(g) and 1(h)).

Screw symmetry
Next, we consider the cases with screw symmetry. In this case, the combination between I symmetry and screw rotation symmetry leads to the mirror symmetry whose mirror plane is offset from the inversion center. For example, we consider the two symmetry operations, {I|000} and {C 2z |00 1 2 } corresponding to the type-I MSG No. 11. These symmetries lead to {M z |00 1 2 }, whose mirror plane does not include the inversion center. In this case, when µ 1 = 2 and the Chern number on k z = 0 plane is zero, the mirror Chern number is nontrivial as discussed in Appendix E. It leads to the emergence of the surface states protected by mirror symmetry, and therefore the mass distribution in this case corresponds to Type 3 in Fig. 1.

Glide symmetry
In this subsection, we consider the cases with glide symmetry. The existence of glide operation invalidates our assumption that the surface orientation specified by n uniquely determines the sign of the mass. For example, let us consider the glide {M y |00 1 2 }. Then, if the mass on the z = 0 surface is positive, the mass on the z = 1/2 surface is negative. Thus, the mass sign depends on a choice of surface terminations. Therefore, we cannot determine the sign of the mass on the z = 0 surface uniquely by glide symmetry.
Next, we consider other surface orientations with such ambiguity of the mass sign. Let (αβγ) denote the Miller index for the surface with this mass ambiguity. It follows that β = 0. The (α0γ) plane can be written as where d is a constant. This plane is transformed into under the glide {M y |00 1 2 }. The mass ambiguity appears when the two surfaces expressed by Eq. (A5) and by Eq. (A6) are parallel but are not equivalent under lattice translation symmetry. Thus, the following equation holds: γ ≡ 1 (mod 2) and β = 0. Thus, the (α, 0, γ) surfaces have such ambiguity if γ is an odd number, and therefore there is an infinite number of surface orientations on which the mass ambiguity arises.
As discussed in the main text, when γ ≡ 0 (mod 2) and α ≡ 1 (mod 2), the (α0γ) plane is invariant under the glide {M y |00 1 2 }, and therefore the mass on this plane is zero. From this, within the glide-mirror plane, two cases for the behavior of the surface mass coexist: the mass on the (α, 0, γ) surface is zero when α ≡ 1 (mod 2) and γ ≡ 0 (mod 2), while the mass is indeterminate when γ ≡ 1 (mod 2). Thus, for the surface normal within the glide-mirror plane, the dependence of the mass on the surface normal vector n is singular. To specify the singular behavior for n perpendicular to the glide plane, we draw broken circles, and we call the cases where the mirror planes in Types 3-8 are replaced with glide mirror as Types 3 -8 (Figs. 2(b-g)).
Appendix B: Proof of C z slab = 0 when N is an even number Here, we show that a Chern number is C z slab = 0 for a slab of layered AFMs with I symmetry andT = C 2z T symmetry with the even number of layers. In the following, N represents a number of the layers. In the following, we show the relation C z slab = −C z slab via the combination of I and an anti-unitary operationT = C 2z T , where T represents a time-reversal operation and C 2z represents a two-fold rotation around z-axis as shown in Fig. 8. Slabs with even N are symmetric under the combination of a unitary operation I and an anti-unitary operationT : where H(k) is the Hamiltonian of the slab with even N . From this, for an occupied state |u n (k) of the Hamiltonian H(k) with an eigenvalues E n , the following relation holds: H(−k)T I |u n (k) =T IH(k) |u n (k) = E nT I |u n (k) .
(B2) Therefore,T I |u n (k) is an eigenstate of the Hamiltonian H(−k), and we can expand where U mn (k) are the matrix elements of a unitary transformation acting on the space of occupied states. Because of the anti-unitarity ofT , we obtain Therefore, the Berry connection is expressed as and then the Berry curvature is written by Therefore, we obtain the relationship C z slab = −C z slab . We note that this proof is based on the existence of the anti-unitary operationT , which interchanges the slab with even N and its inversion partner (see Fig. 8). In other layered AFMs with an anti-unitary operatorT such asT = C 3z T , C 4z T or C 6z T , the Chern number of the even slab is zero. where µ 1 = 0 or 2 (mod 4). Therefore, when µ 1 = 2 in the bulk, the slab system with odd N is a 2D Chern insulator with C z slab = 1 (mod 2), i.e. C z slab = 2M + 1. In such a case, we can choose C z slab = 1 without losing generality. One can simultaneously attach two 2D Chern insulators with the same Chern number on two surfaces of the opposite sides of the crystal, so that I symmetry is preserved 39,41,43 . From this, a Chern number = 2M + 1 can be transformed into a Chern number = 1 while preserving I symmetry. Thus, in the main text, we choose the C z slab = 1. In addition, we consider Chern numbers in the slabs with a finite system size along the x and y directions; C x slab and C y slab . Here we assume the crystal shapes are I symmetric. In this case, we obtain the Chern numbers C x slab and C y slab form the topological Z 4 index µ 1 as where µ 1 is even 39,79 . Therefore, the HOTI with µ 1 = 2 is a 2D Chern insulator in the case with a finite thickness along the x and y directions. Indeed, as shown in Fig. 9(a) and 9(b), the projections of CHSs onto the yz or xz planes form loops corresponding to chiral edge states due to the Chern numbers C x slab ≡ C y slab ≡ 1 (mod 2), in both cases with odd and even N . Therefore, IAHS (for even N ) are topological gapless states characterized by This topological properties of IAHS are different from that of conventional I-symmetric hinge states for odd N , i.e. C x slab ≡ C y slab ≡ C z slab ≡ 1. We note that the relationship C x slab ≡ C y slab ≡ 1 (mod 2) is due to the bulk topology µ 1 = 2, and the 2D Chern numbers remain nontrivial as long as the topological invariant µ 1 in the bulk is preserved. In this appendix, we discuss connection between the mirror Chern numbers and the symmetry-based indicator µ 1 . Here, we consider a 3D magnetic insulator with I symmetry and M z mirror symmetry. First, we define the mirror Chern numbers in the k z = 0 and k z = π sectors as respectively, where C 0 ± and C π ± represent the Chern numbers in the mirror sectors with mirror eigenvalues ±i in the k z = 0 and k z = π sectors respectively. In the following, we prove that the Z 4 symmetry-based indicator µ 1 in an insulator is related with mirror Chern number via where µ 1 = 0, 2. The values µ 1 = 0, 2 means that the bulk is insulating, while µ 1 = 1, 3 corresponds to the Weyl semimetal phase. Equation (D2) means that when µ 1 = 2, one of the two mirror Chern numbers C 0 m and C π m , is an odd number, and the other is an even number, which means emergence of topological surface states on a mirror-symmetric surface.
In the following, we prove Eq. (D2). First, let C m denote the quantity of the r.h.s. of Eq. (D2); Next, we get the relation because in an insulator the total Chern number on the k z = const plane is independent of the values of k z . In addition, from Ref. [89], the following two equations hold: where ξ (±) (Λ i , k z ) is the parity eigenvalues of the occupied states in ± mirror sectors, and Λ i = (k x , k y ) represents 2D TRIM. From Eqs. (D4), (D5) and (D6), the following equation holds: (D7) Next, let N (β) α (α = ±, β = ±) the number of irreducible representations (irreps) with α parity in the β mirror sector. Then we obtain By the way, the M z mirror eigenvalue is common within k z = 0 plane and it is also the case within k z = π plane. Therefore the number of the irreps in the (+) mirror sector, N (+) Similarly, in the (−) mirror sector, the following equation holds: By using the Eqs. (D8), (D9) and (D10), we get the following equation: Then, the C 2z eigenvalue is common between k = (Λ i , 0) and k = (Λ i , π). Therefore, the total number of irreps with the positive eigenvalue +i (+1) of C 2z at (Λ i , 0) for spinful fermions (spinless fermions), is the same as the number with the same eigenvalue at (Λ i , π). By noting C 2z = M z I, we obtain Thus we obtain Eq. (D2).
Appendix E: Connection between mirror Chern numbers and µ1 in the presence of screw symmetry In this appendix, we discuss connection between the mirror Chern numbers and µ 1 in the presence of screw C 2z ≡ {C 2z |00 1 2 } symmetry. Here, we consider a 3D magnetic insulator with I symmetry and screw C 2z symmetry. In addition, the combination between the I symmetry and the screw symmetry leads to a mirror symmetry M z ≡ {M z |00 1 2 }, where mirror plane z = 1 4 does not include the inversion center at (0, 0, 0). In the following, we prove that the mirror Chern number (see Eq. (D1)) with respect to this mirror symmetry is related with the symmetry-based indicator µ 1 via when the Chern number on k z = 0 plane is zero. Equation (E1) means that when µ 1 = 2, topological surface states on a mirror-symmetric surface appear.
In the following, we prove Eq. (E1). First, we consider the plane k z = π. In this case, C 2z operation and M z operation anticommute: Therefore, an occupied state |u n (k) and another occupied state C 2z |u n (k) have the opposite mirror eigenvalues to each other. Here, the number ν of occupied states is an even number. Then the Chern number of |u n (k) is equal to that of C 2z |u n (k) , and these two states are in the opposite mirror sectors. Therefore, we get the equation C π + = C π − . In other words, the mirror Chern number in the plane k z = π is zero: Similarly, I operation and M z operation anticommute, and then an occupied state |u n (k) and another state M z |u n (k) have the opposite parity eigenvalues each other, and they are degenerate. Therefore, the number of the irreps with even parity N + (Λ i , π) is the same as that with odd parity N − (Λ i , π): where Λ i = (k x , k y ) runs through four 2D TRIM. Next, we consider the plane k z = 0. Unlike k z = π plane, I operation and M z operation commute on k z = 0 plane. Let N (β) α (α = ±, β = ±) the number of irreps with α parity in the β mirror sector. Because the M z mirror eigenvalue is common within k z = 0 plane, the following equation holds: In addition, on the line (k x , k y ) = Λ i , the C 2z symmetry is preserved, and the eigenstates are classified into two C 2z sectors with C 2z = ±e −ikz/2 . At the 3D TRIM (Λ i , π), C 2z and I anticommute, and therefore the number of occupied states with C 2z = ±e −ikz/2 is ν/2. Thus we conclude that at the 3D TRIM (Λ i , 0), the number of occupied states with C 2z = ±e −ikz/2 is ν/2. By noting C 2z = M z I, we obtain In addition, from Eqs. (E5) and (E8), we get the following equation: Here we note that the Chern number along the xy plane is even because C = C π + + C π − ≡ Λi N − (Λ i , π) = ν 2 × 4 = 2ν ≡ 0 (mod 2) from Eq. (E4). Therefore, from Eq. (E9), we get which relates the three topological invariants µ 1 , C and C 0 m . In particular, when we assume that the Chern number on k z = 0 plane is C ≡ C 0 + + C 0 − = 0 as we adopted in our classification of the CHSs, it follows that From this equation and Eq. (E3), we obtain Eq. (E1).