Intrinsic ferromagnetic and antiferromagnetic axion insulators in van der Waals materials Mn\emph{X}$_{2}$\emph{B}$_{2}$\emph{T}$_{6}$ family

The MnBi$_{2}$Te$_{4}$ family has attracted significant attention due to its rich topological states such as the quantum anomalous Hall (QAH) insulator state, the axion insulator state, and the magnetic Weyl semimetal state. Nevertheless, the intrinsic antiferromagnetic (AFM) interlayer coupling in MnBi$_{2}$Te$_{4}$ partly hinders the realization of"high-temperature"QAH effect. Here, by using first-principles electronic structure calculations, we design a new class of materials Mn\emph{X}$_{2}$\emph{B}$_{2}$\emph{T}$_{6}$ (\emph{X}=Ge, Sn, or Pb; \emph{B}=Sb or Bi; \emph{T}=Se or Te) based on the \emph{X}$_{2}$\emph{B}$_{2}$\emph{T}$_{5}$ structures rather than the Bi$_{2}$Te$_{3}$ family. We find that each septuple-layer Mn\emph{B}$_{2}$\emph{T}$_{4}$ is sandwiched by two [\emph{X}\emph{T}] layers, which may turn the AFM interlayer coupling into a ferromagnetic (FM) coupling. The calculations specifically demonstrate that \emph{MnGe}$_{2}$\emph{Sb}$_{2}$\emph{Te}$_{6}$, \emph{MnGe}$_{2}$\emph{Bi}$_{2}$\emph{Te}$_{6}$, and \emph{MnPb}$_{2}$\emph{Bi}$_{2}$\emph{Te}$_{6}$ are FM axion insulators, while MnGe$_{2}$Sb$_{2}$Se$_{6}$, MnGe$_{2}$Bi$_{2}$Se$_{6}$, MnSn$_{2}$Sb$_{2}$Te$_{6}$, and MnSn$_{2}$Bi$_{2}$Te$_{6}$ are A-type AFM axion insulators. These seven materials all have an out-of-plane easy axis of magnetization. The Mn\emph{X}$_{2}$\emph{B}$_{2}$\emph{T}$_{6}$ family thus offers a promising platform beyond the MnBi$_{2}$Te$_{4}$ family for the realization of quantized magnetoelectric effect and"high-temperature"QAH effect in future experiments.

Introduction: In recent years, MnBi 2 Te 4 as an intrinsic antiferromagnetic (AFM) topological insulator (TI) has attracted intensive attention [1][2][3][4][5][6][7][8][9][10][11][12], because its unique structural characteristics endow with rich magnetic topological states, including the axion insulators (AXIs), the quantum anomalous Hall (QAH) insulators, and the magnetic Weyl semimetals. The tetradymitetype MnBi 2 Te 4 [13] is composed of Te-Bi-Te-Mn-Te-Bi-Te septuple-layer (SL) blocks stacking along the c axis via the van der Waals (vdW) interaction, which can be regarded as an [MnTe] layer being inserted into the middle of each Bi 2 Te 3 unit with a Te-Bi-Te-Bi-Te quintuple layer (QL), as shown in Figs. 1(a) and 1(b). The ferromagnetic (FM) coupling between intralayer Mn atoms and the AFM coupling between interlayer Mn atoms result in an A-type AFM configuration in MnBi 2 Te 4 , whose easy magnetic axis is along the out-of-plane direction [2,5,6]. Obviously, the topological properties of MnBi 2 Te 4 derive from the parent TI Bi 2 Te 3 , while the magnetism originates from Mn 2+ ions in the [MnTe] layer. On one hand, the intrinsic topological properties of MnBi 2 Te 4 -related ternary chalcogenides (such as MnSb 2 Te 4 , MnSb 2 Se 4 , and MnBi 2 Se 4 ) are very sensitive to the strength of spinorbit coupling (SOC) [5,[14][15][16], since the common origin of TIs is the SOC-induced band inversion [17,18]. On the other hand, due to the intrinsic A-type AFM magnetism in bulk MnBi 2 Te 4 , the zero-field QAH effect appears only in an odd-number SL films [3,6]. For example, it was reported that the zero-field QAH effect was observed at 1.4 K in a five-SL MnBi 2 Te 4 film, and an external magnetic field that enforces the interlayer coupling from AFM to FM further raised the quantization temperature to 6.5 K or even 45 K [3,19]. Therefore, searching * kliu@ruc.edu.cn † zlu@ruc.edu.cn for stoichiometric MnBi 2 Te 4 -related compounds satisfying both the TI properties with strong SOC and the FM ground state with relatively strong interlayer coupling has become one of the main subjects for the realization of "high-temperature" QAH effect. At present, much effort has been devoted to realizing an interlayer FM coupling between the MnBi 2 Te 4 -related layers, such as charge/element doping [20,21], constructing heterojunctions with different d -orbital vdW materials [22][23][24], and inserting Bi 2 Te 3 or Sb 2 Te 3 layers in between the MnBi 2 Te 4 layers to increase the Mn-Mn interlayer distance [25][26][27]. We notice that all these proposals are implemented in the framework of adopting the TI Bi 2 Te 3 family as parent materials. We would like to ask whether there are other TIs as parent materials beyond the Bi 2 Te 3 family that can achieve the intrinsic magnetic TIs with an FM coupling.
In this study, we find that the TI materials X 2 B 2 T 5 (X =Ge, Sn, or Pb; B =Sb or Bi; T =Se or Te) [see Fig. 1(c)], including the TIs Ge 2 Sb 2 Te 5 and Pb 2 Bi 2 Te 5 that were synthesized experimentally and predicted theoretically [28][29][30][31], can serve as the parent materials of Fig. 1(d)]. The MnX 2 B 2 T 6 can be considered as intercalating an [MnT ] layer into the middle of each X 2 B 2 T 5 nonuple-layer block [Figs. 1(c) and 1(d)], which is similar to the case of Bi 2 Te 3 serving as the parent material of MnBi 2 Te 4 [13]. One common feature of Bi 2 Te 3 , MnBi 2 Te 4 , X 2 B 2 T 5 and MnX 2 B 2 T 6 is that each atomic layer in these compounds forms a triangular lattice with only three distinct positions labeled as A, B, and C [see Fig. 1  ily materials by using first-principles electronic structure calculations. The computational details are provided in the Supplemental Material (SM) [39], which includes Refs. [32][33][34][35][36][37][38].
Results: To evaluate the stability of MnX 2 B 2 T 6 family (X =Ge, Sn, or Pb; B =Sb or Bi; T =Se or Te), we have first calculated the phonon spectra of the MnX 2 B 2 T 6 monolayers, in which seven materials MnGe 2 Sb 2 Se 6 , MnGe 2 Sb 2 Te 6 , MnGe 2 Bi 2 Se 6 , MnGe 2 Bi 2 Te 6 , MnSn 2 Sb 2 Te 6 , MnSn 2 Bi 2 Te 6 , and MnPb 2 Bi 2 Te 6 monolayers are found to have no soft modes across the Brillouin zone (BZ), indicating their dynamical stabilities. In comparison, the other five materials have imaginary phonon modes (see Fig. S1 in the SM [39]). Thus, we focus on the seven materials with dynamical stabilities.
We now consider the various magnetic configurations in the monolayer forms of these seven materials and then examine the interlayer magnetic coupling to identify their magnetic ground states. Since the Mn atoms form a triangular lattice in each plane, we consider four typical magnetic structures [see Figs. 1(g)-1(j)], namely FM, single row AFM (SR-AFM), double row AFM (DR-AFM), and ZigZag AFM (ZigZag-AFM) states. Our calculations show that the FM state has the lowest energy among these magnetic states (see Table SI in the SM [39]). Note that the energy is not available for the ZigZag-AFM state, for which the calculations tend to be not converged or converged to the nonmagnetic (NM) state, suggesting that this magnetic structure is unstable. Meanwhile, we have tested different Hubbard U values (U =3, 4, and 5 eV) on the Mn 3d orbitals, and found that the selection of U values has almost no effect on the structural and magnetic properties. Thus, in the following discussion we take U = 4 eV, which is the same as those in Refs. [6,14]. From Table SI, we see that the bond angles of Mn-T -Mn in these seven materials are in the range of 92.1 • -94.8 • (close to 90 • ), hence the FM coupling between the intralayer Mn atoms may be understood from the Goodenough-Kanamori rule [40][41][42]. Furthermore, the calculated magnetic anisotropy energies (MAE) show that their easy magnetization axes all tend to be out-ofplane and the MAE values are comparable to the one of MnBi 2 Te 4 (0.2 meV) [6]. Thus these seven materials all exhibit the intralayer FM coupling with an out-of-plane easy magnetic axis.
The tetradymite-type MnX 2 B 2 T 6 can crystalize in a rhombohedral structure with the space group D 5 3d  Table I. The structural, magnetic, and topological properties of bulk compounds MnGe2Sb2Se6, MnGe2Sb2Te6, MnGe2Bi2Se6, MnGe2Bi 2Te 6, MnSn2Sb2Te6, MnSn2Bi2Te6, and MnPb2Bi 2Te 6, including the lattice constants (a and c) optimized in their respective magnetic ground states, the bond angles of Mn-T -Mn between the two nearest Mn atoms, the atomic magnetic moments (M atom), the energy differences of the A-type AFM state with respect to the FM state (∆EAFM−FM), the numbers of occupied bands with even and odd parities (n + occ ,n − occ ) at the eight inversion-invariant momenta points (Λα) with the inclusion of SOC, and the Z4 invariants.

Structures
Lattice constants Angles (  (No. 166), which is composed of eleven-layer blocks stacking along the c-axis through vdW interaction [see Fig. 1(d)]. We now only need to consider the interlayer magnetic coupling to identify the magnetic ground states for their bulk materials. From the calculated results in Table I [14] is 0.3 meV/Mn lower than that of its FM state. Thus, the bulk compounds MnGe 2 Sb 2 Te 6 , MnGe 2 Bi 2 Te 6 , and MnPb 2 Bi 2 Te 6 host the FM ground states with an outof-plane easy axis of magnetization, which are desirable for the realization of intrinsic "high-temperature" QAH effect. Next, we study the electronic properties of these seven materials. Considering that the X, B, and T in MnX 2 B 2 T 6 respectively represent the same column in the periodic table and have the same valence, they should have similar electronic properties with the same magnetic structures. Here the FM MnGe 2 Sb 2 Te 6 and A-type AFM MnSn 2 Sb 2 Te 6 are selected as examples for illustration, and the electronic properties of other materials can be obtained in the SM [39].
The calculated band structure and projected density of state (PDOS) of the FM MnGe 2 Sb 2 Te 6 are respectively shown in Figs. 2(a) and 2(b). We see that it exhibits metallic characteristics and the bands around the Fermi level are mainly contributed by the p orbitals of Te, Sb, and Ge atoms. In contrast, for the AFM MnSn 2 Sb 2 Te 6 , from the calculated band structure [ Fig. 2(c)] and PDOS [ Fig. 2(d)] we see that it exhibits an insulating behavior and the bands around the Fermi level are also mainly contributed by the p orbitals of Te, Sb, and Sn atoms. The When the SOC is included, the band structures display substantial changes. Figures 3(a) and 3(b) show that the valence bands and the conduction bands of both FM MnGe 2 Sb 2 Te 6 and A-type AFM MnSn 2 Sb 2 Te 6 are inverted at the Γ point around the Fermi level, which implies that they may have nontrivial topological properties. We would like to remind that for centrosymmetric three-dimensional insulators without time-reversal symmetry, the quantized axion angle θ = 0 or π corresponds to the invariant Z 4 =0 (topological trivial) or 2 (topological nontrivial), respectively [44,45]. Because of the presence of the inversion (I ) symmetry in MnX 2 B 2 T 6 family, the parity-based higher-order Z 4 invariant can be adopted to characterize the topological properties of MnX 2 B 2 T 6 with different magnetic configurations. The parity-based Z 4 invariant is defined as [45][46][47] where ξ n (Λ α ) is the parity eigenvalue (+1 or -1) of the nth band at the αth inversion-invariant momenta point Λ α and n occ is the total number of occupied bands. Here Z 4 =1 or 3 indicates a Weyl semimetallic phase, while Z 4 =2 implies an axion insulator (AXI) with a quantized topological magnetoelectric effect (TME) [the axion angle θ = π] in the case of the Chern numbers on all the 2D planes of the BZ being zeros [48]. Thus, we first compute their Chern numbers in the k z = 0 and k z = π planes of these seven materials and find that the Chern numbers in both planes equal to 0. Then, we calculate the number of occupied bands with even and odd parity eigenvalues (n + occ ,n − occ ) at the eight inversion-invariant momenta points (only four of them are distinct) for these seven materials in their respective magnetic ground states. As shown in Table I, we see that all the invariants Z 4 equal to 2 for these seven materials, suggesting that these intrinsic FM MnGe 2 Sb 2 Te 6 , MnGe 2 Bi 2 Te 6 , and MnPb 2 Bi 2 Te 6 as well as A-type AFM MnGe 2 Sb 2 Se 6 , MnGe 2 Bi 2 Se 6 , MnSn 2 Sb 2 Te 6 , and MnSn 2 Bi 2 Te 6 bulk materials are all AXIs with the quantized TME.
As mentioned above, the energy differences between the FM state and the A-type AFM magnetic ground state for MnGe 2 Sb 2 Se 6 , MnGe 2 Bi 2 Se 6 , MnSn 2 Sb 2 Te 6 , and MnSn 2 Bi 2 Te 6 are very small (about 0.02-0.05 meV/Mn), hence a small external magnetic field is sufficient to induce an FM state. The band structures of MnSn 2 Sb 2 Te 6 , MnGe 2 Sb 2 Se 6 , MnGe 2 Bi 2 Se 6 , and MnSn 2 Bi 2 Te 6 bulk materials in the FM states with spins aligned along the c-axis direction calculated with the inclusion of SOC are shown in Fig. 3(c) and Fig. S4 in the SM, respectively. One can see that their energy bands are inverted at the Γ point around the Fermi level. Further, we calculate the number of occupied bands with even and odd parity eigenvalues and obtain their invariants Z 4 still being 2 (see Table SII in the SM). This indicates that all the seven materials are robust AXIs regardless of being ferromagnetic or antiferromagnetic. Note that an AXI can be regarded as a threedimensional gapped system with the pseudoscalar axion angle θ = π and a gapped surface state (SS) at the same time [49], where an additional term S θ = θ 2π e 2 h d 3 xdtE · B related to θ is used to describe the TME. Here E and B are the electric and magnetic fields respectively, e is the charge of an electron, and h is Planck's constant. Now, we have demonstrated that these seven materials all have axion angle θ = π. Next, we take the FM MnGe 2 Sb 2 Te 6 and A-type AFM MnSn 2 Sb 2 Te 6 as examples to show that they both have gapped surface states (SSs). Since these materials are formed by stacking the eleven-layer building blocks along the c axis via the vdW interaction, the (111) plane of their primitive cells (namely, the a-b plane within the vdW gap [see Fig. 1(d)]) is their natural cleavage surface. Thus, we calculated the surface states of this plane for the FM MnGe 2 Sb 2 Te 6 and A-type AFM MnSn 2 Sb 2 Te 6 , as shown in Figs. 4(a-b) and 4(c), respectively. One sees that the A-type AFM MnSn 2 Sb 2 Te 6 does have the gapped SSs [ Fig. 4(c)]. The gap is caused by the interaction between the electrons in the SSs and the spontaneous magnetization [50], which is similar to the case of the surface states of MnBi 2 Te 4 [5,8]. In comparison, the FM MnGe 2 Sb 2 Te 6 appears to have a gapless SS [ Fig. 4(a)], but a careful inspection shows that the gapless SS is not protected by any symmetry. Therefore, a small perturbation will cause it to open an energy gap. We add an onsite energy of 0.1 eV on its surface and find that it does evolve into a gapped SS [see Fig. 4 To gap all the surface states and observe the TME, for the above AFM AXIs, one can apply a small in-plane magnetic field or synthesize a sample without S symmetry [5], where the S = Tτ 1/2 (T is the time-reversal symmetry and τ 1/2 is the half translation operator). For the above FM AXIs, the interference of the gapless hinge modes on the TME can be avoided by designing a sample with a specific trigonal prism geometry to localize its chiral hinge modes on their top surface [51]. Therefore, the MnX 2 B 2 T 6 family provides a series of feasible platform for the long-sought quantized TME.
Discussion: Here, we would like to emphasize the importance and advantages of our current work. First, we break through the framework based on the Bi 2 Te 3 family as the parent material of the intrinsic magnetic TI, and propose a series of intrinsic FM AXIs (such as MnGe 2 Sb 2 Te 6 , MnGe 2 Bi 2 Te 6 , and MnPb 2 Bi 2 Te 6 ) obtained from the X 2 B 2 T 5 family as their parent material. Second, the FM MnGe 2 Sb 2 Te 6 , MnGe 2 Bi 2 Te 6 , and MnPb 2 Bi 2 Te 6 films compared with the A-type AFM MnBi 2 Te 4 film can achieve a comparable quantization temperature under the zero magnetic field with-out the constraint of the number of layers, for which we will discuss in detail in a follow-up work. Third, the MnX 2 B 2 T 6 family can also be extended to a large class of MnX 2 B 2 T 6 /(B 2 T 3 ) n family similar to MnBi 2 Te 4 /(Bi 2 Te 3 ) n family, and Mn can also be replaced with other transition-metal or rare-earth element, thereby it provides a broad platform for studying various magnetic topological states. Finally, we offer a new design scheme for the realization of the intrinsic AXIs, and the idea may also apply to other inversion-preserved vdW TIs. Thus, we look forward to the experimental synthesis of MnX 2 B 2 T 6 family materials in the near future.
Summary: We design a new class of materials MnX 2 B 2 T 6 (X =Ge, Sn, or Pb; B =Sb or Bi; T =Se or Te) based on X 2 B 2 T 5 structures and study the topological properties of these materials by using firstprinciples electronic structure calculations. We find that the interlayer magnetic couplings can be well adjusted by substituting different elements (X, B, and T ) in these systems while maintaining their nontrivial topological properties.
Our calculations specifically demonstrate that MnGe 2 Sb 2 Te 6 , MnGe 2 Bi 2 Te 6 , and MnPb 2 Bi 2 Te 6 are ferromagnetic axion insulators (AXIs), while MnGe 2 Sb 2 Se 6 , MnGe 2 Bi 2 Se 6 , MnSn 2 Sb 2 Te 6 , and MnSn 2 Bi 2 Te 6 are A-type antiferromagnetic AXIs. In addition, they all have an out-ofplane easy axis of magnetization. Thus the MnX 2 B 2 T 6 family materials may offer a promising platform for the realization of the long-sought quantized magnetoelectric effect and the "high-temperature" QAH effect in future experiments.