Generalization of the Nested Wilson Loop Formalism in Topological Dirac Semimetals with Higher-order Fermi Arcs

We generalize the nested Wilson loop formalism, which has been playing an important role in the study of topological quadrupole insulators, to two-dimensional (2D) and 3D nonsymmorphic materials with higher-order topology. In particular, certain 3D Dirac semimetals exhibit 1D higher-order Fermi arc (HOFA) states localizing on hinges where two surfaces meet and connecting the projection of the bulk Dirac points. We discover that the generalized nested Berry phase (gNBP) derived from this formalism is the bulk topological indicator determining the existence/absence of HOFAs, revealing a direct bulk-hinge correspondence in 3D Dirac semimetals. Finally, we study the Dirac semimetals NaCuSe and KMgBi based on first-principles calculations and explicitly show that the change in gNBP adjacent to the Dirac point corresponds to the termination of HOFAs at the projection of Dirac points on the hinge. Our findings not only improve the understanding of the bulk-hinge correspondence in topological Dirac semimetals but also provide a general formalism for studying the higher-order topology in nonsymmorphic systems.

Introduction.-Thebulk-boundary correspondence, which is the novel feature of topological states of matter, relates the nontrivial bulk topology to the presence of gapless states at boundaries [1,2].Recently, the notion of bulk-boundary correspondence has been extended to higher-order forms where topologically protected gapless modes emerge at even lowerdimensional boundaries, such as corners or hinges of crystals [3][4][5][6][7][8][9][10][11][12][13][14].In seminal works [3], Benalcazar et al. have introduced the topological quadrupole insulator (TQI) which features a quantized bulk quadrupole moment and zero-energy corner modes.Based on the nested Wilson loop approach [3], the quadrupole moment in a TQI is determined from the nested Berry phase (NBP), a topological indicator [8] associated with the bands formed by Wannier centers of electrons, which is in analogy to the conventional Berry phase of electronic bands in crystals [15,16].
Recently, certain Dirac semimetals have been shown to exhibit unique higher-order bulkhinge correspondence, i.e., 1D higher-order Fermi arc states localize on hinges where two surfaces meet and terminate at the projection of bulk Dirac points [17][18][19][20][21][22].The HOFA was regarded as a direct consequence of the bulk topology which is characterized by the change in NBP of 2D slices adjacent to the Dirac point in the 3D Brillouin zone (BZ) of Dirac semimetals [17].However, this was limited to the specific cases with fourfold rotational symmetry, which raises the question of whether the topological analysis based on NBPs of BZ slices is still valid for other Dirac semimetals with different symmetries.Moreover, it has been found that some nonsymmorphic crystals exhibit higher-order corner states but only have a specially defined NBP [23][24][25], thus posing an open question of how to identify the bulk topology to furnish the bulk-hinge correspondence for these Dirac semimetals with nonsymmorphic symmetries.
In this work, we generalize the nested Wilson loop formalism to nonsymmorphic systems with higher-order bulk-boundary correspondence.The generalized NBP (gNBP) derived from this formalism at 2D BZ slices indexed by k z is proved to be a bulk topological indicator that directly determines the existence/absence of HOFAs on hinges of Dirac semimetals with nonsymmorphic symmetries.Based on first-principles calculations, we further study two Dirac semimetals, NaCuSe and KMgBi, and explicitly demonstrate that the change in gNBP across the Dirac points is associated with the termination of HOFAs at the projection of Dirac points on the hinge.
Generalization of nested Wilson loop.-Wefirst introduce the generalization of the nested Wilson loops and NBP.For a 2D BZ in the k x -k y plane or any 2D slices with fixed k z in a 3D BZ, the Wilson loop along the k x direction with the base point k is defined as, where is the Berry connection of occupied bands, and P is the path-ordering operator.Diagonalizing these Wilson loop operators, one arrives at where eigenstates x (k y ) which is only a function of k y , form Wannier bands over the 1D BZ k y ∈ (−π, π] and obey the identification {ν j x (k y )} ≡ {ν j x (k y ) mod 1}.Due to certain symmetry constraints, such as mirror reflections, these Wannier bands can be naturally divided into two groups, i.e., two Wannier sectors, which are separated from each other by Wannier gaps in the Wannier band spectrum [3].The key insight behind the nested Wilson loop formalism is that, being gapped, the Wannier sectors can carry their own topological invariants.Here we generalize this idea by dividing the occupied energy bands into N s (≥ 2) subspaces according to the generic splitting of Wannier sectors constrained by symmorphic and/or nonsymmorphic symmetries, where the superscript s denotes the Wannier sectors, with j ∈ {1, 2, labeling the bands within a sector.Here we assume that N occ is an integer multiple of N s , otherwise the Wannier spectrum is always gapless [4].The nontrivial nature of each Wannier sector is determined by the generalized Wannier-sector polarization (WSP), Similar procedures of Eqs. ( 1)-( 4) are also applicable to the k y direction, which bring up the Wilson loop operator W y,k , Wannier bands ν j y (k x ), and generalized WSP p ν s y x along the x direction.In practice, we numerically determine the Wilson loop operators by discretizing the BZ and the detailed procedure is thoroughly explained in Supplementary Material (SM) [26].Under symmetry constraints, these generalized WSPs would take quantized values, which are expected to indicate the band topology, as discussed later.For convention, we choose the i-th WSP as an indicator so that the gNBP θ (s) = 2π(p ) are zeros in the trivial atomic limit.
Symmetry constraints and quantization.-It is noted that Ref. [3] is the symmorphic case for 2D lattices with reflection symmetries M x : x → −x and M y : y → −y, which enforce Wannier bands to occur in pairs The generalization of NBP is demanded in the cases of nonsymmorphic symmetry, where conventional NBP θ (2) β vanishes, but the existence of higher-order topological features indicates the urgent need for an essential topological indicator.In general, a nonsymmorphic symmetry operator of a layer group can be expressed as {O α |δ ι }, where O α is a two-fold rotation C 2α or a reflection M α with α = x or y, and δ ι takes fractional values (in unit-cell units): δ x = ( 1 2 , 0, 0), δ y = (0, 1 2 , 0), and δ = ( 1 2 , 1 2 , 0).For example, under glide reflections G α = {M α |δ}, the Wannier bands pair up differently, Hence, due to the constraints of G x and G y , the Wannier bands always occur in quadruple: . This provides a natural splitting of Wannier bands into four sectors (i.e., N s = 4): ) with s = 1, 2, 3, 4 when N occ = 4n.Consequently, the quartern WSPs obey Although the constraints from G x and G y alone are not enough to quantize p ν s x y itself, a composite WSP of two quarter Wannier sectors take quantized values which depends only on the parity of n: p 2 n mod 1.This has been used to characterize the so-called "anomalous" quadrupole insulators [24,25], which cannot undergo a topological phase transition to trivial insulators unless the fundamental nonsymmorphic symmetries are broken.
More importantly, we found that under some combination of nonsymmorphic and/or symmorphic symmetries, the generalized WSP would take quantized values, which signals the higher-order topology when the conventional WSP vanishes.For example, the p4/nmm layer group includes both reflection M α with α ∈ {x, y} and off-centered rotation [27].Because the out-of-plane reflection symmetry M z would not affect the Wilson loop along in-plane directions, the constraints generated by {C 2α |δ} are the same as those generated by glide reflection G β [see Eqs. ( 5) and ( 6)], indicating a quadrisection of Wannier bands.M α further quantizes the quartern WSP, which still satisfy the constraint of Eq. ( 6).
It is worth mentioning that the generalized WSP applies not only to orthogonal systems with nonsymmorphic symmetries but also to systems with non-orthogonal primitive cells.
For example, in C 3 -or C 6 -symmetric systems, one can redefine a non-primitive rectangular cell to restore the periodicities along two orthogonal directions.A glide symmetry emerges by combining reflection with fractional translation {E| 1 2 , 1 2 , 0} in units of the redefined cell [28].Then, the generalized WSP can be calculated analogically.We systemically considered all 80 layer groups, which describe the spatial symmetry of 2D spinless systems [29].The results are summarized in Table I (see SM for more details [26]).
In particular, the generalized WSP for layer groups can also extend to describe the higher-order topology of 2D insulating planes of 3D BZ indexed by k z = 0, π which are usually described by magnetic layer groups.In general, the time-reversal T and reflection Hence, symmetries that keep the generalized WSP invariant in generic slices (k z = 0 or π) are in-plane symmetries g xy inherited from its 3D bulk and the composite symmetries combining g xy , T , and M z .In fact, a one-to-one mapping between the allowed symmetries of a k z slice and a layer group can be constructed by regarding M z T in the former to M z in the latter.This is exemplified by considering a minimal model for the nonsymmorphic space group P 4/nmm that describes a 3D Dirac semimetal with a pair of Dirac points on the k z axis, as shown in Fig. 1.Details of the tight-binding model are presented in SM [26]).The symmetry of a generic k z = 0, π slice are M x , M y , and G z T , which give rise to the nonsymmorphic symmetry operator Since M z T only reverses the Wannier band ν α (k β ) → ν α (−k β ) but does not affect Wannier gaps within a k z slice, the k z slice exhibits a similar quadrisection of Wannier bands and quantization of generalized WSP as the layer group p4/nmm, which has been studied above.
Specifically, the four Wannier sectors: ν x (k y ), 1/2 − ν x (−k y ), −ν x (k y ), 1/2 + ν x (−k y ) are related by M x , G z T and M x • G z T , as shown Fig. 1(c,d).Although Wannier bands exhibit distinct behaviors for k z -slices below or above a Dirac point, the conventional NBP θ (2)   vanishes for k z -slices on both sides of the Dirac point, indicating its failure in diagnosing band topology.
Remarkably, with increasing k z across a Dirac point, the quartern WSP undergoes a sudden jump by 1/2, implying a topological phase transition for 2D k z -slice subsystems.
Meanwhile, we find that the Dirac semimetal exhibits higher-order Fermi arcs (HOFAs) terminating at the projection of the bulk Dirac points on side hinges.Such HOFAs can be viewed as topological bulk-hinge correspondence for Dirac semimetals [17].Viewing each k z slice of the 3D BZ as an effective 2D subsystem, the Dirac point is equivalent to the critical point between 2D trivial and higher-order topological insulators with corner states.
Therefore, our proposed generalized WSP serves as a topological indicator to determine the presence/absence of HOFAs.Our approach is also consistent with the previous classification of Dirac points in Refs.[30,31] where the termination of HOFA is signaled by the change of filling anomaly for 2D k z slices on either side of the Dirac point [26].Because of the preserved PT symmetry, every band is doubly degenerate.It is noteworthy that the s-dominated band is lower than the Se-p band at Γ, resulting in an inverted band structure.Since the two bands belong to Γ 6 and Γ 7 irreducible representations with respect to C 4z in the little group of Γ-Z, they can simply cross each other without opening a gap at the Fermi level, which forms a time-reversed pair of Dirac points at (0,0,±k D ) near the Γ point.Interestingly, the Dirac points can be shifted along the k z axis by external strains that respect the C 4z symmetry.For better observation of the HOFAs, we increase the separation between two Dirac points ∆k = 2k D by applying a uniaxial tensile strain of 6% along the c axis, as shown in Fig. 2(c).Below, we utilize the strained system to illustrate the topological properties, unless otherwise specified.

Material
To reveal the nontrivial higher-order topology in NaCuSe, we calculate the Wannier bands for k z slices on either side of the Dirac point.Due to the C 4z symmetry, it is sufficient to just consider ν x (k y ).For generic slices (k z = 0, π, or ±k D ), four Wannier sectors are separated by Wannier gaps, as shown in Fig. 2(d,e).The higher-order topology is identified by the generalized WSP p x ] has a nontrivial value of π for the nonzero values of k z between two Dirac points (0 < |k z | < k D ).The sudden change of θ (4) y indicates that k z slices with hinge-localized HOFA states occur between the projection of bulk Dirac points, which agrees with our analysis of the change in filling anomaly between k z slices adjacent to the Dirac point [26].The dramatic change of Wannier bands around the Dirac points can also be verified by directly scanning ν x (k z ) adjacent to k z = ±k D in the k y = 0 plane [26].It is noted that Wannier gaps vanish in the T -invariant k z = 0 or π slices, which makes the generalized WSP ill-defined.Instead, the Z 2 topological invariants are well-defined in these planes and can identify the band topology [32][33][34].It is found that  Z 2 = 1 for k z = 0 plane which exhibits the topology of a 2D TI, whereas Z 2 = 0 for the k z = π plane.This identifies NaCuSe as a topologically nontrivial Dirac semimetal according to Ref. [27,35].
Figure 3 shows our analysis of the surface and hinge spectra of the strained Dirac semimetal NaCuSe.As shown in Fig. 3(a,b), there are some surface states on the (100) surface, which, however, are not necessitated to connect to surface projections of the bulk Dirac points [19,36].These states are not a topological consequence of the bulk Dirac points, but rather appear on the Γ-Ȳ line solely due to the nontrivial Z 2 topology of the k z = 0 slice.In the spectrum of a single hinge in a right-angle prism-shaped nanorod [Fig.3(c,d)], we observe HOFA states in the vicinity of k z = 0 (i.e., Γ) that connect the hinge projections of the bulk Dirac points to the projections of 2D surface TI cones at k z = 0.This hinge mode is consistent with our analysis of the generalized WSP from the bulk electronic structure.
Thus we confirm that the bulk-hinge correspondence in Dirac semimetals can be diagnosed by the change in generalized WSP.
z ) parameterized as a function of k z .As shown in Fig.2(f), the gNBP component θ(4)

FIG. 1 ., 1 2 ,
FIG. 1.(a) Schematic illustration of a Dirac semimetal with a pair of Dirac points in space group P 4/nmm.The HOFAs exist between the projections of two Dirac points on side hinges.The symmetries that keep generalized WSP invariant in slices at fixed k z = 0, π can be mapped to the layer group p4/nmm by regarding M z T in the former as M z in the latter.(b) Each k z slice is viewed as an effective 2D subsystem with the symmetry of layer group p4/nmm.The composite operator combing G z = {M z |δ} and T leads to a fractional transition {E| 1 2 , 1 2 , 0}. (c),(d) Wannier bands of two k z slices at opposite sides of a Dirac point, where the quartern WSPs are 0 and 1/2, respectively.Four Wannier sectors are related by M x , G z T , and M x • G z T symmetries.

FIG. 2 .
FIG. 2. (a) Crystal structure of NaCuSe with P 4/nmm symmetry, where Na, Cu, and Se atoms occupy the 2c, 2a, and 2c ′ Wyckoff positions, respectively.(b) Orbital-resolved band structure of pristine NaCuSe, where the valence and conduction bands cross along Γ-Z forming a Dirac point near the Γ point.(c) Band structures along Γ-Z of NaCuSe under a tensile strain of 6% along the z-axis.(d) and (e) are Wannier bands on k z = 0.04π and k z = 0.20π slices which are on opposite sides of the Dirac point (k z = k D ).The four Wannier sectors are marked in different colors.(f) The gNBP component θ (4) y of k z slices.
(k z ) of k z slices are trivial on either side of Dirac points, which is different from previous results based on a simplified k • p Hamiltonian obtained by fitting to the first-principles band structures.Instead, we find that only the gNBP undergoes a change of (π, π) when k z passes through the Dirac point, in agreement with the HOFAs terminating at the projected Dirac point.This again validates the necessity for the generalization of the nested Wilson loop formalism in Dirac semimetals with nonsymmorphic space groups.Conclusion.-We generalize the nested Wilson loop formalism to indicate the higherorder topology in 2D and 3D systems with nonsymmorphic symmetries.Due to symmetry constraints, Wannier bands are naturally split into multiple sectors and the gNBP of each sector can take quantized values, which serves as a new topological indicator to diagnose higher-order band topology.In particular, we find that the existence/absence of HOFAs in 3D Dirac semimetals is determined by the change in the gNBP of k z slices adjacent to Dirac points.Employing first-principles calculations, we demonstrate the HOFAs and gNBP in Dirac semimetals NaCuSe and KMgBi.This work was supported by the National Key R&D Program of China (No. 2021YFA1401600), the National Natural Science Foundation of China (Grant No. 12074006), and the top-notch student training in basic disciplines program 2.0 research project (Grant No. 20222005).H.Z. and W.D. acknowledge support from the Basic Science Center Project of NSFC (Grant No. 52388201) and the Beijing Advanced Innovation Center for Future Chip (ICFC).The computational resources were supported by the high-performance computing platform of Peking University.

TABLE I .
Layer groups that admit higher-order topology with quantized gNBP as a topological indicator.