Site Mixing for Engineering Magnetic Topological Insulators

The van der Waals compound, MnBi$_2$Te$_4$, is the first intrinsic magnetic topological insulator, providing a materials platform for exploring exotic quantum phenomena such as the axion insulator state and the quantum anomalous Hall effect. However, intrinsic structural imperfections lead to bulk conductivity, and the roles of magnetic defects are still unknown. With higher concentrations of same types of magnetic defects, the isostructural compound MnSb$_2$Te$_4$ is a better model system for a systematic investigation of the connections among magnetic, topology and lattice defects. In this work, the impact of antisite defects on the magnetism and electronic structure is studied in MnSb$_2$Te$_4$. Mn-Sb site mixing leads to complex magnetic structures and tunes the interlayer magnetic coupling between antiferromagnetic and ferromagnetic. The detailed nonstoichiometry and site-mixing of MnSb$_2$Te$_4$ crystals depend on the growth parameters, which can lead to $\approx$40\% of Mn sites occupied by Sb and $\approx$15\% of Sb sites by Mn in as-grown crystals. Single crystal neutron diffraction and electron microscopy studies show nearly random distribution of the antisite defects. Band structure calculations suggest that the Mn-Sb site-mixing favors a FM interlayer coupling, consistent with experimental observation, but is detrimental to the band inversion required for a nontrivial topology. Our results suggest a long range magnetic order of Mn ions sitting on Bi sites in MnBi$_2$Te$_4$. The effects of site mixing should be considered in all layered heterostructures that consist of alternating magnetic and topological layers, including the entire family of MnTe(Bi$_2$Te$_3$)$_n$, its Sb analogs and their solid solution.


I. INTRODUCTION
Structural imperfection in crystalline materials can have profound effects on the electronic, magnetic, optical, mechanical, and thermal properties. For topological insulators, the surface state is protected against nonmagnetic lattice defects due to spin-momentum locking. Magnetic dopants have been purposely introduced to topological insulators to induce magnetism, which breaks time reversal symmetry. This approach led to the first experimental observation of the quantum anomalous Hall effect (QAHE) in Cr 0.15 (Bi 0.1 Sb 0.9 ) 1.85 Te 3 films [1]. This complex chemical formula highlights the importance of identification and fine manipulation of lattice defects in realizing the exotic properties of topological materials. Because of the random distribution of Cr in the Bi layer, however, the QAHE was only observed at extremely low temperatures of 30 mK, even though the onset of magnetic ordering occurs at ≈15 K [1].
MnBi 2 Te 4 , a natural heterostructure of magnetic (MnTe) and topological (Bi 2 Te 3 ) ingredients, in principle should be an ideal platform for observing the QAHE and axion behavior since it is a stoichiometric compound instead of a random alloy. However, MnBi 2 Te 4 , like any * liuyh@ornl.gov † yanj@ornl.gov crystalline compound, will have defects. As shown in Fig. 1(a), MnBi 2 Te 4 consists of septuple layers of Te-Bi-Te-Mn-Te-Bi-Te stacked along the crystallographic c-axis [2]. Below T N =25 K, the high spin Mn 2+ (d 5 , S = 5/2) ions order ferromagentically in each septuple layer which then couple antiferromagnetically with neighbouring layers forming the so-called A-type antiferromagnetic (AFM) structure [3,4]. Therefore, in addition to those lattice defects observed in Bi 2 Te 3 [5][6][7][8][9][10], magnetic defects are expected in MnBi 2 Te 4 . The question then becomes what type and what concentration of defects can be tolerated to preserve the entangled quantum ground state? How do these defects impact the magnetism and the observation of the QAHE? These questions are very general and apply to any material proposed for observation of the QAHE. Experimental investigations [4,[11][12][13][14] reported antisite disorders of Mn Bi (Mn at the Bi site with one negative charge following Kroger-Vink notation), Bi . M n (Bi at Mn site with a positive charge), Bi T e and likely Mn vacancies in MnBi 2 Te 4 crystals or films. Density functional theory (DFT) calculations [15] suggest the lattice mismatch between MnTe and Bi 2 Te 3 sheets facilitates the formation of Mn-Bi site mixing. These lattice defects lead to the bulk conductivity and affect the spin wave spectra [16,17]. However, it is unknown whether the Mn ions at Bi site order magnetically, how the magnetic defects affect the magnetism and whether they are related to the diverse surface states resolved by angle-resolved photoemission (ARPES) studies [3,13,[18][19][20][21][22][23]. Furthermore, ample experimental evidence suggests these magnetic defects also exist in other MnTe(Bi 2 Te 3 ) n members with n>1 [24][25][26][27][28][29][30][31][32]. Therefore, it is critical to understand the formation, distribution, consequences, and manipulation of the magnetic defects in this family of compounds.
MnSb 2 Te 4 is a model system for such a systematic investigation of the connections among magnetism, topology, and lattice defects. MnSb 2 Te 4 is isostructural to MnBi 2 Te 4 , and these two compounds have the same types of defects [15]. However, the concentration of lattice defects in MnBi 2 Te 4 is only a few percent [4], which is difficult to systematically study. In contrast, MnSb 2 Te 4 tends to have higher concentrations of magnetic defects because the difference of ionic size and electronegativity between Mn and Sb is much smaller than that between Mn and Bi in MnBi 2 Te 4 . From crystal growth point of view, MnSb 2 Te 4 crystals can be grown easily in a wide temperature range which makes it possible fine tune of the concentrations of lattice defects and hence the magnetism by varying the growth temperatures. The growth of MnBi 2 Te 4 has been quite challenging and the crystallization occurs in a narrow temperature window of ≈10 • C [4,13]. Finally, while MnSb 2 Te 4 single crystals are reported to order antiferromagnetically [33], polycrystalline samples are reported to have ferrimagnetic septuple layers that are coupled ferromagnetically due to the presence of Mn at Sb site [34]. It is unknown whether this complex magnetic structure is inherent to the polycrystalline sample or the discrepancy between single crystal and polycrystalline samples results from different amount of magnetic defects and signals their essential roles. Of particular interest is the origin of the ferromagnetic (FM) inter-septuple-layer coupling, which might be employed to fine tune the magnetism of MnBi 2 Te 4 for the observation of QAHE at elevated temperatures.
In this work, we, therefore, perform a systematic study of the magnetic defects in MnSb 2 Te 4 single crystals with different magnetic ground states and tunable magnetic ordering temperatures. Our results show that the randomly distributed Mn-Sb antisite defects favor a FM interlayer coupling but are detrimental to the band inversion necessary for a non-trivial band topology. Our results imply that Mn Bi ions in MnBi 2 Te 4 order ferromagnetically below T N =25 K but they are antiferromagentically coupled to Mn × M n ions in each septuple layer. Partial substitution of Bi by magnetic ions in MnBi 2 Te 4 might be a valid approach toward a FM inter-septuplelayer coupling and quantum Hall effect at zero magnetic field.

II. EXPERIMENTAL DETAILS
MnSb 2 Te 4 single crystals were grown out of an Sb-Te flux [4]. After homogenizing the starting materials at 900 • C for overnight, the mixture was furnace cooled to a certain temperature in between 620 • C and 640 • C and kept at the selected temperature for two weeks. The fixed-temperature growth was employed to obtain uniform crystals because the lattice defects are sensitive to the growth temperatures and starting compositions. The crystals were then separated from Sb-Te flux by decanting. In this work, we focus on the crystals grown at three different temperatures of 620 • C, 630 • C, and 640 • C. The growth temperature determines the nonstoichiometry and site mixing and thus the magnetic order. We also tried crystal growths starting with different ratio of MnTe:Sb 2 Te 3 , which also affects the nonstoichiometry and site mixing in MnSb 2 Te 4 and enables a fine tuning of the magnetic ordering temperature up to 50 K (Fig. S4 in Supporting Information).
Elemental analysis on cleaved surfaces was performed using either the energy dispersive (EDS) or the wavelength dispersive (WDS) spectroscopy techniques. The EDS measurement was performed using a Hitachi TM-3000 tabletop electron microscope equipped with a Bruker Quantax 70 energy dispersive x-ray system. The WDS measurement was performed using a JEOL JXA-8200X electron microprobe analyzer instrument equipped with five crystal-focusing spectrometers for waveength dispersive x-ray spectroscopy. Magnetic properties were measured with a Quantum Design (QD) Magnetic Property Measurement System in the temperature range 2.0 K≤ T ≤ 300 K. The temperature and field dependent electrical resistivity data were collected using a QD Physical Property Measurement System.
High resolution single crystal neutron diffraction data were collected at 300 K at TOPAZ at the Spallation Neutron Source (SNS) to determine the average nuclear structures. The integrated Bragg intensities are obtained using the 3D ellipsoidal Q-space integration method and are corrected for background using MAN-TID software [35]. Data reduction including Lorentz and absorption corrections as well as spectrum, detector efficiency, data scaling, and normalization are carried out with the ANVRED3 program [36]. The structure models were analyzed with the SHELX prgoram [37]. Low temperature single crystal neutron diffraction experiments were carried out at beamline CORELLI [38] at SNS and the Four-Circle Diffractometer (HB3A) at the High Flux Isotope Reactor (HFIR) to determine the average magnetic structure. Possible magnetic structures were investigated by the representation analysis using the SARAh program [39], as well as the magnetic space group approach, where the maximal magnetic space groups were obtained from the MAXMAGN program [40]. Nuclear and magnetic structure refinements were carried out with the FullProf Suite program [41]. Additionally, the temperature dependence of the magnetic order and potential diffuse scattering in all three samples were investigated at CORELLI.
Scanning transmission electron microscopy (STEM) observations were performed on a Nion UltraSTEM100, equipped with a cold field-emission gun and a correc-tor of third-and fifth-order aberrations, operated at the accelerating voltage of 100 kV. About 2 × 2 µm 2 thin specimens of MnSb 2 Te 4 were prepared by focused-ionbeam (FIB), and subsequently by ion milling at a low voltage of 1.5 kV and liquid nitrogen temperature for 20 min. High-angle annular dark-field (HAADF)-STEM images were collected with a probe convergence angle of 30 mrad and an inner collection angle of 86 mrad. Scanning tunneling microscopy (STM) measurements were performed at 48 K in an Omicron LT-STM with base pressure 1 × 10 −10 mbar. Electrochemically etched tungsten tips were characterized on clean Au(111) surface before STM experiments. Single crystals of MnSb 2 Te 4 were cleaved in situ at room temperature and immediately inserted into a cold STM head.
Band structure with spin-orbit coupling (SOC) in density functional theory [42,43] (DFT) have been calculated with PBE [44] exchange-correlation functional, a plane-wave basis set and projected augmented wave method as implemented in VASP [45,46]. To account for the half-filled strongly localized Mn 3d orbitals, a Hubbard-like U [47] value of 3.0 eV is used. The (2×2×2) MnSb 2 Te 4 rhombohedral unit cell is sampled with a Monkhorst-Pack [48] (6×6×3) k-point mesh including the Γ point and a kinetic energy cutoff of 270 eV. For band structure calculations, the experimental lattice parameters have been used with atoms fixed in their bulk positions. The supercell has also been fully relaxed in PBEsol [49] exchange correlation functional with U for total energy difference. An antiferromagnetic(AFM)-like anisotropic magnetic susceptibility (see Fig. 1(b)), χ, for the crystals grown at 620 • C is in contrast to a FM-like temperature dependence (see Fig. 1(c)) for those grown at 640 • C. The magnetic susceptibility for both batches shows a similar temperature dependence when the external magnetic field is applied perpendicular to the crystallographic caxis. However, when the magnetic field is applied along the c-axis, χ c in Fig. 1(b) decreases sharply when cooling below 19 K; while χ c in Fig. 1(c) shows a rapid increase upon cooling below the magnetic ordering temperature. The former is similar to that observed for MnBi 2 Te 4 suggesting an AFM interlayer coupling [4,[11][12][13][14]33]. The latter suggests a FM interlayer coupling, as previously observed in polycrystalline samples [34]. This is also supported by the field dependence of magnetization shown in Figs. 1(d) and 1(e). A well defined spin flip transition is observed in Fig. 1(d) when the field is applied along the c-axis. This spin flip transition is absent in Fig. 1(e), also suggesting a FM interaction along the c-axis. When the magnetic field is applied perpendicular to the c-axis, a similar field dependence is observed for both crystals; however, the magnetic field at which the magnetization tends to saturate decreases from about 25 kOe in Fig. 1(d) to 10 kOe in Fig. 1(e). The saturation moment obtained by extrapolating M (H) curve to H = 0 is 1.95(2) and 1.98(2) µ B /Mn, respectively. For both crystals, the anisotropic field and temperature dependencies of magnetization suggest the ordered magnetic moments are along the c-axis.

A. Distinct magnetic properties
The magnetic measurements suggest that the sign of the interlayer interaction is tunable by the growth temperature. The crystals grown below 620 • C show the AFM-like χ(T ), as in Fig. 1(b); and those grown above 625 • C show a FM-like behavior, as in Fig. 1(c). While all AFM-like crystals order magnetically at the same T N , the magnetic ordering temperature of FM-like crystals can be tuned in the temperature range 24 K-50 K by controlling the growth parameters as shown in Fig. S4 in Supporting Information. As described later, the magnetic ordering temperature and saturation moment depend on the nonstoichiometry and distribution of Mn. In the following text, we refer to those three crystals that are thoroughly investigated as T N 19K, T C 24K, and T C 34K, respectively, with T N indicating AFM-interlayer coupling, T C for FM-interlayer coupling, and the number as the magnetic ordering temperature.
B. Site mixing and magnetic structure from neutron single crystal diffraction The anisotropic magnetic properties suggest that the interlayer coupling can be either FM or AFM although in both cases the ordered moment is aligned along the c-axis. To understand the different magnetic behaviors and determine the detailed magnetic structures, we performed single crystal neutron diffraction experiments in a wide temperature range using multiple diffractometers. High-resolution diffraction data collected at 300 K at beamline TOPAZ confirmed that the average structure can be well described by the space group R3m for both types of crystals ( Fig. S1 in Supporting Information), similar to previous reports on powder [34] or single crystal samples [33,50]. Neutron diffraction is sensitive to the Mn-Sb mixing because of the high scattering contrast between Mn and Sb nuclei. for T N 19K, T C 24K, T C 34K, respectively (Table S1-S4 in Supporting Information). The refined elemental ratio agrees with that obtained from elemental analyses as summarized in Table S6 in Supporting Information. Interestingly, with increasing growth temperatures, Low temperature diffraction data were collected at CORELLI to investigate the magnetic structure and potential diffuse scattering. Datasets for refinement were collected for T N 19K and T C 24K at both 30 K and 6.5 K. The diffraction patterns collected above magnetic ordering temperature are similar for both crystals (e.g. Fig. S2(a) in Supporting Information). However, they are quite different in the magnetically ordered state at 6.5 K. The diffraction pattern in Fig. 2(a) for T C 24K is similar to that collected at 30 K but the intensity of some peaks is enhanced as highlighted by the white arrows ( Fig. S2 in Supporting Information), suggesting a magnetic wavevector of k F M = (0 0 0). T N 19K (see Fig. 2(b) shows a completely new set of peaks at the half-L positions below T N , suggesting a magnetic wavevector of k AF M = (0 0 1.5). Fig. 2(c) shows the temperature dependence of intensities of selected Bragg peaks which confirms the magnetic ordering wavevectors as well as the magnetic ordering temperatures as found from magnetic and transport measurements.
The refinement on the CORELLI data shows the same amount of Mn-Sb site mixing as determined from TOPAZ measurements (Table S5 in Supporting Information). The absence of structured diffuse scattering patterns suggests that the antisite defects do not induce significant local lattice distortions, and they are either randomly distributed or have a correlation length shorter than the probing limiting ≈ 1 nm. There is no magnetic Bragg scattering contribution for (0 0 L)-type peaks for both T N 19K and T C 24K (data not shown), suggesting that the ordered moment is along the c-axis. This is consistent with the anisotropic magnetic properties presented above. Figures 2(c) and 2(d) show the magnetic structures determined from the low temperature diffraction datasets for T C 24K and T N 19K, respectively. As shown in Fig. S3 in Supporting Information, the magnetic order of Mn Sb ions has to be considered leading to the ferrimagnetic septuple layer for both magnetic structures. Both Mn × M n and Mn Sb ions carry significant ordered moments and they couple antiferromagnetically in each septuple layer. At 6.5 K, the ordered moment of ≈4.0µ B /Mn for Mn × M n is slightly larger than ≈3.5µ B /Mn for Mn Sb . A distinct difference between the magnetic structures shown in Figs. 2(d) and 2(e) is the sign of the inter-septuple-layer coupling. A FM interlayer coupling shown in Fig. 2(d) is in contrast to an AFM one in   [4,11], Mn Sb in MnSb 2 Te 4 exists as a dark triangle in STM topography due to the depression in the three nearest-neighbor Te atoms in the topmost plane right above the Mn Sb . The density of Mn Sb is estimated to be ≈10.6% in T C 24K, slightly larger than ≈8.4% in T N 19K. The ratio is about 1.26, consistent with our neutron scattering results. To explore the distribution of Mn Sb , the radial distribution function analysis is performed after locating each Mn Sb . The radial distribution function g(r) describes how the density of defects varies as a function of distance, by calculating the normalized density of defects within a distance between r− 1 2 dr and r+ 1 2 dr from a reference defect, where dr is the binsize. Here two binsizes of 0.5 nm and 1 nm are used. As shown in Figs. 3(f) and 3(g), g(r) is smaller than 1 at a short distance of 0.5 nm, and fluctuates around 1 at larger distances, indicating some Mn Sb -Mn Sb repulsion with a short correlation scale. In contrast, the clustering of defects would result in g(r) much larger than 1 at short distances, which is absent in our analysis. A careful investigation of our STM images suggests ≈0.3% of Sb T e defects in both T C 24K and T N 19K crystals.

D. Electronic structure calculations
To understand the effects of site mixing on the magnetic order and electronic structure, we performed DFT calculations. We first constructed a (2×2×2) rhombohedral supercell at stoichiometry with the Mn-Sb sitemixing ratio of 50 Figure 4(a) shows the most stable configuration in the (2×2×2) supercell (labelled as AS3 in Table S7 and Fig. S8 in Supporting Information) with the complex magnetic structure. Additionally we also considered two configurations in the larger (3×3×2) supercell with the Mn-Sb site-mixing ratio of 56/44 in the central layer and 22/78 in the Sb layer, where the central Mn layer has a mixture of Mn and Sb in each row, unlike the alternating rows in (2×2×2). As listed in Table S7, after full relaxation in PEBsol+U [47,49], a FM interlayer coupling is preferred for all the antisite configurations, in a distinct contrast to the ideal case without site-mixing. Such a preference for all the configurations also holds when SOC is included during the full relaxation. As a reference, for the ideal MnSb 2 Te 4 without site-mixing, the A-type AFM structure is 3.48 meV/f.u. more stable than the FM order for PBEsol+U with relaxation. The (2×2×2)-AS3 complex magnetic configuration in Fig. 4(a) with an FM interlayer coupling is 0.31 meV/f.u. more stable than that with an AFM interlayer coupling. Although in the above supercells the variation for composition and configuration is limited compared to the experimental compositions, these total energy differences confirm that the magnetic coupling between the septuple layers changes the preference from AFM to FM with more Mn at Sb site, in qualitative agreement with experimental observation. We further studied the effects of Mn-Sb site mixing on the electronic structure. First, our calculations show that the ideal MnSb 2 Te 4 with an A-type AFM order has a small gap [34,50,51] without band inversion and is topologically trivial. In contrast, the ideal MnSb 2 Te 4 with a FM order is topologically non-trivial supported by the Weyl point (black arrow) along Γ-Z in Fig. 4(b) due to the band inversion between Sb-derived conduction and Te-derived valence band. Stabilizing FM in MnSb 2 Te 4 is promising to realize magnetic Weyl points, similar to EuCd 2 As 2 [52,53]. These results are consistent with previous reports [34,50,51]. We then introduced Mn-Sb site mixing as discussed above and the electronic band structures of the most stable configuration in the (2×2×2) supercell with AFM and FM interlayer couplings are shown in Figs. 4(c) and 4(d), respectively. As shown in Fig. 4(c), each band is still doubly degenerated because the supercell retains inversion symmetry and the effective timereversal symmetry [54]. The combined time-reversal operation and half-translation along the c-axis is the same as in the ideal A-type AFM configuration. But the band gap is increased to 0.3 eV, the opposite direction for having a band inversion at Γ. With a FM interlayer coupling (see Fig. 4d), the band double degeneracy is lifted and the top valence band and bottom conduction band move toward each other reducing the band gap to 0.2 eV, albeit still no band inversion. Furthermore, from the total density of states (see Fig. S9 in Supporting Information) for all the site-mixing supercell configurations considered with both AFM and FM-like coupling, there is a bulk band gap of 0.2 eV or beyond. Compared to the FM configuration of ideal MnSb 2 Te 4 , in contrast to the increased band gap at Γ and Z points, the gap at L point in Fig. 4(d) is reduced for MnSb 2 Te 4 with significant site mixing. These band structure results indicate that although site-mixing in MnSb 2 Te 4 is beneficial to stabilize a FM interlayer coupling, it is detrimental to retain the band inversion needed for non-trivial topology. A high magnetic field is expected to ferromagnetically align Mn moments at both Mn and Sb sites. As shown in Fig. S10 in Supporting Information, the calculation with all moments ferromagnetically aligned suggests still no band inversion although the band gap is slightly reduced compared to that in Fig. 4(d). To justify the calculations, Fig. S11 in Supporting Information shows the scanning tunneling spectra data of T C 24K and T N 19K at 4.5 K. Both samples have a bulk band gap of about 0.4 eV. The flat and diminished density of state within the bandgap indicates the absence of in-gap states and trivial topology of the band structures. The calculated band gap in these supercell configurations agrees with the scanning tunneling spectra (STS) data, considering the underestimation of band gap of the PBEsol exchange-correlation functional.

A. Effects of site mixing on the magnetic and electronic properties of MnSb2Te4
Our results show that magnetic defects can have a dramatic effect on the magnetic ordering in MnSb 2 Te 4 , and Mn-Sb site mixing favors a FM interlayer coupling. The anisotropic magnetic properties shown in Fig. 1 demonstrate that the inter-septuple-layer coupling can be FM or AFM depending on the crystal growth parameters. Our neutron diffraction and microscopy measurements found a significant amount of Mn-Sb site mixing with nearly random distribution in all crystals investigated in this work. We noticed that similar Mn-Sb site mixing is also observed in a polycrystalline sample with a FM interlayer coupling by neutron powder diffraction [34] and in a single crystal sample by single-crystal x-ray diffraction [50]. The observation of Mn-Sb site mixing in MnSb 2 Te 4 samples synthesized differently by different groups suggest that the presence of Mn-Sb mixing is a general phenomena in MnSb 2 Te 4 , possibly due to the small formation energy of antisite defects [15]. By investigating the anisotropic magnetic properties, concentration and distribution of Mn-Sb site mixing, this work develops a direct correlation between the magnetic ground states and magnetic defects. Other intrinsic lattice defects, such as Sb , T e and Te . Sb , might also exist with a smaller concentration and mainly affect the transport properties. Therefore, the formation of the complex magnetic structure and the sign change of inter-septuple-layer coupling result from the occupation of Mn at Sb sites. We noticed that with increasing growth temperature, both Mn × M n and Mn Sb increase in concentration, and therefore does the total Mn content. The difference between T C 24K and T N 19K suggests a threshold concentration of 13% Mn Sb , above which the inter-septuple-layer interaction becomes FM. With further increasing the total Mn content, the magnetic ordering temperature increases to about 50 K as illustrated in Fig. S4 in Supporting Information. Therefore, both the sign of interlayer coupling and the magnetic ordering temperature are tunable by a careful control of the concentration and distribution of magnetic ions in the lattice.
The modification of the magnetic ground state results from the extra competing interactions introduced by Mn Sb . The energy difference between FM and AFM states is small in both ideal MnBi 2 Te 4 and MnSb 2 Te 4 . Therefore, the magnetic ground state can be sensitive to external perturbations. The competing interactions in MnBi 2 Te 4 were studied previously using inelastic neutron scattering [16]. Similar competing interactions are expected in ideal MnSb 2 Te 4 . The presence of Mn Sb introduces extra competing magnetic interactions. Considering a FM order is always observed in TM -doped (TM =Mn, Cr, and V) Bi 2 Te 3 and Sb 2 Te 3 , the interaction between neighbouring Mn Sb and Mn Sb ions right across the van der Waals gap is expected to be FM. This FM interaction increases in magnitude with increasing amount of Mn Sb and competes with the AFM interaction in between Mn × M n and Mn × M n in neighbouring septuple layers and eventually leads to the sign change of interlayer magnetic coupling. Other interactions between further neighbouring Mn ions will also contribute to the overall interlayer magnetic interactions but are possibly less important. Mn Sb magnetic defects also introduce extra exchange interactions in each septuple layer. Both 90 degree and 180 degree Mn-Te-Mn superexchange interactions are active and contribute to the overall interaction between Mn Sb and Mn × M n . While the partial contribution of different exchange paths to the total exchange are to be investigated, the experimentally observed ferrimagnetic arrangement in each septuple layer suggests an AFM interaction between Mn Sb and Mn × M n . As described later, this AFM interaction is rather strong.
Mn-Sb site mixing favors a FM interlayer coupling but is detrimental to the band inversion. Our DFT calculations considered different antisite configurations using the (2×2×2) supercell and found all antisite configurations prefer a FM interlayer coupling. This is further confirmed by using a larger supercell (3×3×2). This FM interlayer coupling is preferred for the realization of a Weyl state. Unfortunately, our band structure calculations suggest a sizable band gap of >0.2 eV regardless of the sign of interlayer magnetic interaction or the antisite configuration. STS measurement further confirms the sizable gap in our crystals with significant amount of Mn-Sb site mixing. The site-mixing induced gap opening is also reproduced in a most recent study by Wimmer et al [55]. In their DFT calculations, they considered the effect of 20% Mn-Sb site mixing on the band structure and found a gap of about 0.2 eV which makes MnSb 2 Te 4 to be topological trivial. In addition, Wimmer et al. also considered the effect of 5% Mn-Sb antisite mixing on the band structure of FM MnSb 2 Te 4 to find that the band inversion and Weyl point are preserved. This interesting result indicates that it is likely to tune the magnetism by introducing Mn-Sb (or Mn-Bi) site mixing in MnSb 2 Te 4 (or MnBi 2 Te 4 ) without losing the band inversion as suggested below.

B. Indications for MnTe(Bi2Te3)n
Since about 3% Mn Bi ions were observed in MnBi 2 Te 4 single crystals by different groups, one naturally asks (1) whether those 3% Mn Bi ions order magnetically forming ferrimagnetic septuple layers as observed in MnSb 2 Te 4 , (2) whether it is possible to tune the magnetic ground state, and thus the topological properties, by controlling the concentration and distribution of magnetic defects in MnBi 2 Te 4 , and (3) whether the site mixing plays any role in other n>1 compounds in MnTe(Bi 2 Te 3 ) n family with more complex stacking of the septuple and quintuple layers. As discussed below, Mn Bi ions likely order magnetically as observed in MnSb 2 Te 4 and the magnetism of Mn Bi , in either the septuple or the quintuple layers, should be considered in all MnTe(Bi 2 Te 3 ) n compounds.
We noticed that the growth dependent magnetic behavior as shown in Fig. 1 is absent in MnBi 2 Te 4 . All previous work reported an AFM order with T N =25 K for MnBi 2 Te 4 crystals. This consistency might result from the narrow crystallization temperature window and/or the limited concentration of magnetic defects due to the large difference of ionic size and electronegativity between Mn and Bi. Considering possible AFM alignment of Mn Bi and Mn × M n , the field dependence of magnetization can provide more valuable information.
As proposed by Murakami et al. [34], the ferrimagnetic arrangement of Mn moments in each septuple layer of MnSb 2 Te 4 provides a straightforward explanation for the large difference between the ordered and saturation moments. With the ordered moment at 6.5 K and the occupancy both determined from neutron single crystal diffraction, the average moment within each septuple layer is 1.52(4) and 1.65(8)µ B /Mn at 6.5 K for T N 19K and T C 24K, respectively (Table S6 in Supporting Information). These values are smaller than the saturation moment determined at 2 K from magnetic measurements but in reasonable agreement. The magnetization at 2 K does not saturate even in an applied magnetic field of 120 kOe, indicating a strong magnetic coupling between Mn × M n and Mn Sb . A field-induced transition to a state with all Mn moments polarized is expected at a much higher magnetic field and this deserves further study.
One important indication of the above analyses is possible magnetic order of Mn Bi ions in MnBi 2 Te 4 . The saturation moment of MnBi 2 Te 4 is ≈3.8 µ B /Mn estimated from magnetic measurements at 2 K [4,16,33], which is smaller than the ordered moment of 4.04(13) µ B /Mn at 10 K [4] or 4.7(1) µ B /Mn at 4.5 K [56] determined from neutron diffraction. The amount of Mn Bi is ≈3% in our crystals [4]. With the assumption that Mn × M n and Mn Bi ions are antiferromagnetically aligned with the same ordered moment of 4.7 µ B , the net moment of each septuple layer is about 4.1 µ B , in reasonable agreement with the experimentally measured saturation moment at 2 K. It should be noted that 2% Mn on Bi sites is enough to induce a long range FM order in Bi 2 Te 3 . From these points of view, the magnetic structure of MnBi 2 Te 4 and its field dependence should be carefully revisited. Considering the similarity between MnBi 2 Te 4 and MnSb 2 Te 4 , a similar complex ferrimagnetic structure is expected for MnBi 2 Te 4 if Mn Bi ions forms a long range magnetic order below T N =25 K. The doping dependence of magnetization in MnBi 2−x Sb x Te 4 was previously carefully studied [33]. The gradually suppressed saturation moment with increasing Sb doping signals the increasing amount of Mn at Bi/Sb site. The evolution with doping of the magnetic and topological properties should be revisited in this system by considering the doping-dependent site mixing and the magnetic ordering of antisite Mn ions. The magnetic order of antisite Mn ions will impact the surface magnetism and, of particular interest are whether and how this magnetic order couples to the topological surface states which deserve further investigation. Similar studies are also relevant for MnTe(Bi 2 Te 3 ) n compounds with n>1 considering the presence of Mn Bi in both the quintuple and septuple layers. STM studies observed 2-3% Mn Bi ions in both the quintuple and septuple layers in MnBi 4 Te 7 crystals [30,32].
Besides the natural antisite defects, one can purposely introduce magnetic ions, such as V, or Cr, at Bi site to tune the magnetism and even changing the sign of interseptuple-layer magnetic interaction in MnBi 2 Te 4 . Realization of such a complex magnetic structure without losing the band inversion might be a valid approach toward QAHE in MnBi 2 Te 4 . Considering the narrow crystallization temperature window for the flux growth of MnBi 2 Te 4 out of Bi-Te flux, vapor transport growth of bulk crystals or MBE growth of thin films might have a better control of the concentration of magnetic ions on Bi sites. On the other hand, reducing the amount of site mixing might lead to quantized transport properties at even higher temperatures than reported in MnBi 2 Te 4 flakes [57][58][59].

V. SUMMARY
In summary, we have demonstrated the importance of understanding and controlling the defect concentration and distribution in any candidate magnetic topological insulators. In particular, we have shown that Mn-Sb site-mixing can have a dramatic effect on the magnetic ground state and electronic structure of MnSb 2 Te 4 . These antisite defects favor a FM interlayer coupling. While an ideal FM MnSb 2 Te 4 crystal hosts Weyl points, site-mixing is detrimental to the desired nontrivial band topology. Our work highlights the importance of further investigating lattice defects in the isostructural MnBi 2 Te 4 and related compounds, where Mn-Bi site mixing has been observed although by a much reduced amount. How the magnetic defects Mn Bi in MnTe.nBi 2 Te 3 affect the surface magnetism and interact with the topological surface states deserve further study. Single crystal neutron diffraction data were collected at 300 K using the beamline TOPAZ at the Spallation Neutron Source (SNS) to determine the average nuclear structure. Low temperature measurements were performed at the beamline CORELLI at SNS and the Four-Circle Diffractometer (HB-3A) at the High Flux Isotope Reactor (HFIR) to determine the average magnetic structure.

Nuclear structure Refinement
Previous studies reported the R3m space group for MnSb 2 Te 4 from powder diffraction experiments [33,34]. With the high resolution single crystal neutron diffraction data collected at TOPAZ (300 K), we examined the crystal structures of all three MnSb 2 Te 4 single crystals. Figure S1 shows the refinement results using R3m for T N 19K, T C 24K and T C 34K. Lowering the crystal symmetry from R3m to its subgroup R3m or C 2/m does not significantly improve the refinement; rather there are high correlated atomic positions and atomic displacement parameters, suggesting that a lower symmetry is unnecessary to describe the average structure in the presence of significant Mn-Sb site mixing. The R3m space group is thus used for all structure refinements below. Table S1 shows the comparison of the structural refinements on the 300 K TOPAZ data assuming different chemical formula. All have the same fitting quality. However, the refinements suggest that all three samples have significant amount of Mn-Sb site mixing with slight differences in the numerical values. Importantly, they all found more Mn ions on both Mn and Sb sites for samples with a ferromagnetic interlayer coupling. Detailed structural refinement results are shown in Tables S2, S3, S4 for T N 19K, T C 24K and T C 34K, respectively. With increasing Mn content, a-lattice parameter decreases slightly while c-lattice is less affected. Figure S2 shows the neutron scattering patterns in the [H 1 L] plane of the T C 24K sample collected at (a) 30 K and (b) 6.5 K. The observed Bragg peaks match the scattering condition -H+K+L = 3n, as required by the space group R3m for the nuclear reflections. The pattern collected at 6.5 K is also presented in the main text and it is presented here to highlight the change cooling below T C .   (2)   The diffraction pattern collected at 6.5 K is similar to that collected at 30 K but the intensity of some peaks is enhanced as highlighted by the white arrows, suggesting

magnetic structures: ordered moments on Mn Sb
Reciprocal space mapping from the CORELLI data found no highly structured diffuse scattering signal, which suggests that MnSb 2 Te 4 remains in the R3m space group at low temperatures. Low-temperature datasets were collected for T N 19K and T C 24K. Partially due to the unavoidable background scattering from the sample environment, the datasets collected at both CORELLI and HB-3A contain much fewer peaks than the 300 K TOPAZ datasets. However, the average nuclear structures determined from the CORELLI and HB-3A data for T N 19K and T C 24K suggest similar amounts of Mn/Site site mixing but with higher uncertainties in comparison to the TOPAZ results. Table S5 shows the site mixing and ordered moments for T N 19K refined using the formula A from neutron diffraction data collected at three different beamlines. Interestingly, to achieve satisfactory refinements of the magnetic structure, one has to consider the ordered moments on Mn Sb , as shown in Fig. S3 for both T N 19K and T C 24K. For T N 19K, the ordered moments are 3.71(2) µ B per Mn M n and 3.46(8) µ B per Mn Sb using the chemical formula A and 3.95(2) µ B per Mn M n and 3.18(7) µ B per Mn Sb using the chemical formula C. For T C 24K, the ordered moments are 4.08(7) µ B per Mn M n and 3.5(2) µ B per Mn Sb for the chemical formula A and 4.12(7) µ B per Mn M n and 3.3(2) µ B per Mn Sb for the chemical formula C. The determined ordered moments using different chemical formulas are slightly different but agree with each other. We thus use formula A in the main text to describe the chemical composition and occupancy. Table S6 summarizes the nonstoichiometry, site occupancy, and magnetic moments for all three crystals.   Figure S4a shows the temperature dependence of magnetic susceptibility of MnSb 2 Te 4 synthesized using different ratios of starting elements. All three batches are homogenized at 1000 • C for 12 hours and then annealed at 640 • C for a week. The atomic ratio of the starting composition is detailed in the figure. Changing the Mn-Sb ratio is expected to vary the magnetic properties. MnSb 2 Te 4 crystals grown in this manner are rather small and the magnetic measurements were performed using a cluster of small crystals with a random orientation. The results suggest that T C can be tuned up to around 50 K. We noticed that there is a small fraction of unknown magnetic impurity for the composition starting with Mn:Sb=1.3:1.7. Further increasing Mn content in the starting materials increases the content of the impu-rity phase. Figure S4b shows the magnetization measured at 2K in applied magnetic fields up to 70 kOe. It is interesting that all samples have a magnetization around 2 µ B /Mn at 2 K and 70 kOe. While detailed nonstoichiometry and atomic occupancy need to be determined, it is reasonable to expect that they determine the ordering temperature and saturation moment. The sample with T C around 50 K has a smaller magnetization possibly due to the presence of the unknown magnetic impurity. Figure S5 shows the temperature and field dependence of the in-plane electrical resistivity, ρ xx , measured in the temperature range 2 K ≤ T ≤ 300 K with the electrical current flowing in the ab-plane. All compounds show a metallic conducting behavior in the paramagnetic state. ρ xx shows an abrupt enhancement cooling across T N for T N 19K. In contrast, a cusp sitting around T C was observed for T C 24K and T C 34K, which is similar to that observed for MnBi 2 Te 4 . All compounds show a negative magnetoresistance in magnetic fields up to 80 kOe applied along the crystallographic c-axis.

C. Transport properties
The Hall resistivity was measured at multiple temperatures in the temperature range 2 K -300 K. A linear field dependence is observed at all temperatures for all samples. Figure S6 shows the data collected at 50 K and 250 K. The Hall coefficient shows little temperature dependence below room temperature. With the assumption that a single band dominates the Hall signal, the coefficient at 250 K gives a carrier density of 2.8×10 20 cm −3 , and 3.7×10 20 cm −3 , and 6.0×10 20 cm −3 TABLE S6. Nonstoichiometry, site occupancy, and magnetic moments for all three crystals. The chemical formula is written as (Mnp1Sb1−p1)(Sb1−p2Mnp2)2Te4, where Te occupancy was fixed to be 1 and p1 and p2 are the site occupation of Mn atoms at Mn and Sb sites, respectively, determined from the 300 K TOPAZ data. The ordered moments (in µB/Mn) are determined from the 6.5 K CORELLI data. The average moment within each septuple layer in the ferrimagnetic state was estimated as (m M n@M n × p1 − m M n@Sb × 2p2)/(p1 + 2p2).  (2) 1.98 (2) for T N 19K, T C 24K, and T C 34K, respectively.
D. STEM Figure S7 shows the intensity variation in Mn atomic columns along (001) planes indicating the presence of atomic mixing. The intensity profiles shown in Fig. S7 b are obtained from the representative atomic planes of Te and Mn labeled in (a). Obviously, Mn atomic columns show a larger intensity variation than Te columns. This agrees with both our diffraction data and previous DFT calculations [15] that there are significant amount of site mixing at the Mn site but not at the Te site.
E. DFT Figure S8 shows the atomic configurations for half of the supercell of MnSb 2 Te 4 with different site-mixing configurations. Also shown are two configurations in the larger (3×3×2) supercell with a smaller site-mixing ratio. Table S7 shows the energy difference between the fully relaxed FM-like and AFM-like magnetic order calculated in PBEsol+U and PBEsol+U+SOC [47,49]. All the supercell configurations with site-mixing prefer the FM-like magnetic order. Figure S9 shows the total density of states (DOS) near E F of the FM-like and AFM-like magnetic orders with different site-mixing configurations. Figure S10 shows the band structure assuming all magnetic moments on Mn ions at different sites are aligned ferromagnetically. The band gap is reduced, but still no band inversion is expected.

F. STS
To validate the electronic structure from DFT calculations, we measured scanning tunneling spectra (STS) of T N 19K and T C 24K at 4.5 K. As shown in Fig. S11, both samples have a bulk band gap of about 0.4 eV, with valence band maximum at 0.2 eV above E F and conduction