Distinct Topological Surface States on the Two Terminations of MnBi$_4$Te$_7$

The recent discovered intrinsic magnetic topological insulator MnBi2Te4 have been met with unusual success in hosting emergent phenomena such as the quantum anomalous Hall effect and the axion insulator states. However, the surface-bulk correspondence of the Mn-Bi-Te family, composed by the superlattice-like MnBi2Te4/(Bi2Te3)n (n = 0, 1, 2, 3 ...) layered structure, remains intriguing but elusive. Here, by using scanning tunneling microscopy (STM) and angle-resolved photoemission spectroscopy (ARPES) techniques, we unambiguously assign the two distinct surface states of MnBi4Te7 (n = 1) to the quintuple-layer (QL) Bi2Te3 termination and the septuple-layer (SL) MnBi2Te4 termination, respectively. A comparison of the experimental observations with theoretical calculations reveals the diverging topological behaviors, especially the hybridization effect between magnetic and nonmagnetic layers, on the two terminations: a gap on the QL termination originating from the topological surface states of the QL hybridizing with the bands of the beneath SL, and a gapless Dirac-cone band structure on the SL termination with time-reversal symmetry. The quasi-particle interference patterns further confirm the topological nature of the surface states for both terminations, continuing far above the Fermi energy. The QL termination carries a spin-helical Dirac state with hexagonal warping, while at the SL termination, a strongly canted helical state from the surface lies between a pair of Rashba-split states from its neighboring layer. Our work elucidates an unprecedented hybridization effect between the building blocks of the topological surface states, and also reveals the termination-dependent time-reversal symmetry breaking in a magnetic topological insulator, rendering an ideal platform to realize the half-integer quantum Hall effect and relevant quantum phenomena.


For
≥ 1 , the hybrid structure creates distinct electronic structures on separate terminations due to the interplay of topology and magnetism at the interfaces of magnetic TI (MnBi2Te4) and non-magnetic TI (Bi2Te3) layers. Local probe approaches to the scattering processes [24][25][26] on different terminations of the van der Waals heterostructure are crucial to determine the spin configuration of the topological surface states and advance the theoretical understanding of the heterostructure engineering. Moreover, despite the evidence of the band structure in MnBi2Te4/(Bi2Te3)n series from ARPES and transport studies, so far there has been little experimental work on probing the robustness of these states against scattering, which is one of the key properties towards the application of topological devices.
Here, we present the first local probe STM measurements of MnBi4Te7 (n = 1) in both the real and momentum spaces. Combined with the observed ARPES band dispersion, we unambiguously assign the observed electronic band structures to the MnBi2Te4 septuple layer (SL) and Bi2Te3 quintuple layer (QL) terminations. Together with the input of ARPES (both regular and spin-resolved) and theoretical calculations, further investigation on the scattering process by quasiparticle interference (QPI) patterns unveils the spin configuration of their topological surface states.
Unlike the surface state of a conventional TI (e.g., Bi2Se3) that is simply localized at the surface layer with attenuation into the bulk, the surface states of MnBi4Te7 shows strong hybridization between the magnetic layer and nonmagnetic layer at both terminations. For QL termination, a gap is formed due to the hybridization between the topological surface bands of the topmost QL and the bands from the neighboring SL layer, hiding the magnetism-induced gap near the Dirac point.
For SL termination, our results suggest that a restoration of time-reversal symmetry occurs on SL termination, possibly due to the magnetically disordered surface. The spin texture of the topological surface bands shows a Rashba-splitting band from the QL underneath and a strongly warped band from SL, the hybridization of which contributes to the flower shaped QPI patterns.
Our findings of such diverging topological behaviors on the two terminations of MnBi4Te7 strongly rely on the hybridization of bands from different building blocks of magnetic TI and nonmagnetic TI, providing new insights in the surface-bulk correspondence of magnetic TIs and guidance to the heterostructure engineering in the emerging intrinsic magnetic topological systems. Fig. 1 illustrates the topographic images obtained by STM, where the two terminations are identified unambiguously. As shown in Fig. 1b, the crystal naturally cleaves at the QL termination of Bi2Te3 and the SL termination of MnBi2Te4 with step heights of 1.01 nm and 1.37 nm respectively. Comparing the zoom-in views of the QL (Fig. 1c) and SL (Fig. 1f) terminations, much more surface defects can be found on the SL termination. Even on the surface defect-free area, QL termination shows less corrugation, indicating lower bulk defect beneath the topmost layer ( Fig. S2). Two types of major defects are found on the SL surface, categorized as bright and dark spots, the former of which is almost absent on the QL surface. To determine the origin of the defects, we obtain the atomic resolution images of the Te-terminated QL ( Fig. 1 d-e) and SL ( Fig.   1 g-h) surfaces at different bias voltages. On both terminations, the dark defects merge from triangularly placed holes to one triangular dark spot when the bias voltage is shifted from 0.5 V to -1 V, while the Te atoms from the topmost layer remains intact. We thus determine that these are

II. SURFACE TOPOGRAPHY
MnBi anitsite defects (Mn replacing Bi at the second atomic layer), as seen in previous STM studies on Mn-doped Bi2Te3 [27]. The bright spots are found to be slightly larger atoms within the surface lattice, as shown in Fig. S3, possibly attributed to BiTe antisite origin (Bi replacing some of the top layer Te) [28]. The density of the detected MnBi anitsite defects is about 3.0% on the QL termination and 2.1% on the SL termination. Note that this is only for the second atomic layer as STM is surface sensitive. As shown in Fig. S2, on top of the MnBi defects on both terminations, there is a slight rightward shift (~10 meV) of dI/dV spectra compared with the defect free area, indicating a local p-type doping from this antisite defect. The migration of magnetic atoms (Mn), illustrated in Fig. 1a, may induce complicated surface magnetic order/disorder on both terminations, especially for the QL layer, as stated in previous studies that such amount of magnetic dopant is sufficient enough to induce long range ferromagnetic order in magnetically doped topological insulators [25,29,30].

III. ELECTRONIC STRUCTURES
We now focus on the defect free regions to obtain our averaged dI/dV spectra by scanning tunneling spectroscopy (STS). The results, as shown in Fig Fig. 2d) is endorsed by a V-shaped dip at the SL STS whose minimum locates at approximately -280 meV (red arrow in Fig. 2a). Therefore, by comparing with the STS spectra, we experimentally ascertain the assignment of the QL and SL terminations in the ARPES measurements for the first time.
We next perform first-principles calculations of MnBi4Te7 slabs to gain a better understanding of the electronic structures of the two terminations. For the QL termination, we assume that the local moment of the SL layers near the surface remains the same as that in the bulk, i.e., A-type AFM with out-of-plane spin orientation confirmed by neutron diffraction measurements [31]. The resulted band structure in Fig. 2e shows that the Dirac cone of the surface states is gapped by the magnetic proximity from the second SL layer, merging into the valence band at ~400 meV below the Fermi level (see the green box in Fig. 2e). More importantly, an indirect gap near the Dirac point energy occurs. The layer projection of the Bloch wavefunctions clearly shows that there is a band inversion between the conduction and valence bands, which is dominated by the topmost QL and the second SL layer, respectively. Therefore, the indirect gap is caused by the hybridization effect between these two layers. This is in sharp contrast to the conventional TI or magnetic-doped TI where the surface states around the Fermi level are governed by the surface layer, with attenuation into the bulk. It is also worth noting that other ARPES studies on the QL termination reveal a Λ-shaped intensity within the abovementioned hybridization gap [19,20]. With the assumption of A-type AFM spin configuration, such an in-gap state can be interpreted as a hybridization effect between different orbitals contributed by the surface QL and its neighboring SL, respectively.
For the MnBi2Te4 SL termination (Fig. 2f), the experimental A-type AFM order with outof-plane moment will inevitably open a sizeable gap of around 60 meV at the surface state ( Fig.   S4), inconsistent with our ARPES observation showing a gapless Dirac cone. In real materials, surface magnetism can be different from the bulk due to various reasons including surface reconstruction, disorder, or spin canting, etc. [32][33][34], which essentially governs the magneticrelated topological behaviors. Especially, considerable amount of MnBi anitsite defects are spotted (as discussed in Fig. 1) on both terminations, which may further complicate the surface magnetism.
Considering some possible magnetic orders on the surface, we suggest a topmost nonmagnetic SL with disordered spin to be the most likely scenario, giving rise to a restoration of the time reversal symmetry on the SL termination. As shown in Fig. 2f, by assuming a nonmagnetic SL surface layer, we perform density functional theory (DFT) calculation and obtain almost linear dispersion with a negligibly gapped Dirac cone (gap size less than 10 meV), originating from the proximity effect of the third MnBi2Te4 layer, in good agreement with our ARPES results (Fig. 2d). Unlike the QL termination, the projection onto layers shows that the linear Dirac bands are mainly contributed by the surface SL layer. However, the strong hybridization between QL and SL layers also takes place as early as 150 meV below the Dirac point inferable from the contours of constant energy data shown in Fig. S9. Moreover, such hybridization can be captured by our quasiparticle interference patterns at higher energies (about 200 meV above the Dirac point).

IV. SPIN-SELECTIVE QUASIPARTICLE INTERFERENCE
Having established the distinct band dispersion at the two terminations, now we carry out CCEs in the ARPES measurement. Fig. 3a and 3e shows typical CCEs of ARPES data for the QL and SL terminations at ~ 50 meV below the Fermi energy, respectively. Comparing with the calculated JDOS patterns from ARPES data ( Fig. 3b and 3f), the FT-STS maps ( Fig. 3d and 3h) display clear suppression of scattering intensity along Γ � − K � directions. Previous FT-STS studies on topological insulators have suggested that the scattering process can be greatly affected by the spin texture of the topological surface states, as helicity only allows interference to occur when there is a finite spin overlap between the two scattering states [24,25,37,38]. On the QL termination, a hexagonally warped shape of surface band structure is observed (Fig. 3a), accompanied by suppressed Γ � − K � scattering in the QPI pattern. This resembles the warpinginduced attenuation of Γ � − K � scattering in Bi2Te3, where out-of-plane spin component can be introduced in between the warped corners, suggesting that the spin texture on QL termination can be understood in a similar manner.
Compared with the case of the QL termination, the QPI pattern of SL is more intricate. We first look into the results from spin-resolved ARPES for possible spin texture guidance at the SL termination. Fig. 2b  The data is a mixed result of both the SL and QL terminations due to the millimeter-sized spatial resolution of the spin ARPES device. Interestingly, the electronic states of MnBi4Te7 shows an up-down-up-down spin configuration above the Dirac point energy (-280 < E < 0 meV), resembling a typical Rashba-like spin splitting. We conclude such a spin pattern comes from the SL termination for the following reasons: (i) the k-space locations of the spin-polarized bands in Fig.   2b match those for the SL termination (Fig. 2d) to a better degree than those for the QL termination ( Fig. 2c). Specifically, the spin pattern at moderate binding energies (i.e., Eb ~ 0.2 eV) spreads to a wider range of ΓM than the QL bands do (dashed curves in Figs. 2b and 2d). (ii) The inner ring in the QL termination (Fig. 3a) has been assigned as a bulk electronic state [13], where spin polarization is assumed to be far less polarized than that of surface states. Therefore, our spin-  Supplementary Fig. S12. We find that a direct gap is reproduced in the QL termination due to the combined effect of the hybridization Δ and exchange field (Fig. S12a), while a gapless Dirac cone beneath the Rashba bands generally captures the feature of SL termination (Fig. S12c).
We next use the obtained spin textures in Fig. S12 to calculate the spin-selective JDOS and compare with the observed QPI.
The spin configurations for the QL and SL terminations are indicated by the arrows in Fig.   4a and b, respectively. We found that (i) the QL termination carries a spin-helical Dirac state with hexagonal warping, which gives rise to the canted spin texture along z direction; (ii) on the SL termination, a Rashba-split surface state comprises a pair of concentric circular bands with antiparallel spin helicity, between which lies another strongly canted helical state whose spin being antiparallel to the outer ring. Note that for the SL termination, hybridization can be observed when the strongly canted flower shape band crosses the outer circular band, as shown in Fig. S12. Consequently, as illustrated by Fig. 4a and b, the major scattering channel forming the edge of the petals in the QPI pattern can only be originated from qSL along Γ � − M � with aligned spins for the SL termination, while for QL termination, the majority of allowed scattering occurs for wavevector qQL. Detailed process of finding qSL and qQL is described in Fig. S11. We then overlay the dispersion of the wavevectors qQL and qSL obtained in the QPI patterns with the energy dispersion expected in the spin-selective JDOS calculated from ARPES data (Fig. S8-9). The plot shows good agreement between QPI and ARPES below Fermi level for both terminations,

V. CONCLUSION
To summarize, we explore the distinct topological surface states of the two terminations of MnBi4Te7 by ARPES and STM. Compared with theoretical calculations, a gapless Dirac electronic structure is observed on the magnetic SL termination, whereas the nonmagnetic QL termination is found to be gapped because of the hybridization between different orbitals of the neighboring TI building blocks, implying different surface magnetism on the two terminations. Despite the microscopic defects of the magnetic atoms, the scattering process on both terminations can be understood by the topological surface states and the newly discovered spin configurations. The robust topological surface states remain present up to an energy regime far above the Fermi energy on both terminations, showing great tolerance to disorders. Our results provide insight not only to the impact of surface magnetism on the topological surface states, but also the potential of hybridization effect in the heterostructure engineering in magnetic TI systems. Further surface sensitive magnetic measurements or quantum transport study may be able to directly probe the detailed surface magnetic ordering on the two terminations and the related exotic quantum states.

APPENDIX: METHODS
The single crystal was prepared by flux growth method and confirmed by first single crystal X-ray diffraction data and the subsequent magnetic and transport measurements. The surface band structures of both terminations are calculated by density functional theory.
We use the projector-augmented wave (PAW) pseudopotentials [40] with the exchange-correlation of Perdew-Burke-Ernzerhof (PBE) form [41] and GGA+U [42] approach within the Dudarev scheme as implemented in the Vienna ab-initio Simulation Package (VASP) [43,44].    (a-b) Schematic of CCEs resulting in the edges of q vectors observed in the FT-STS maps along � − � direction, arrows mark the spin orientation on these topological surface states: on the QL termination (a), the circular shape surface band is slightly warped for E > -100 meV, causing canted spin along z, thus resulting in qQL marked by the blue arrows; on the SL termination (b), for E > -150 meV, Rashba splitting contributes antiparallel helicity for the inner and outer rings in the CCE map (marked as blue and red arrows), where the outer ringhybridizes with a strongly warped surface band sandwiched in between. The middle flower-shaped band has the same helicity as the inner circular bands to fit the intensity along � − � direction in the QPI map, leading to qSL marked by the yellow arrow. (c) Dispersion of the q vectors calculated from the ARPES data (solid lines) on the QL (blue) and SL (orange) terminations, and the q vectors obtained from the FT-STS data (orange stars for QL and purple circles for SL). The original CCE and FT-STS maps are shown in supplementary Figures S6, S7, S13 and S14.