Topological phase transition in a generalized Kane-Mele-Hubbard model: A combined quantum Monte Carlo and Green's function study

We study a generalized Kane-Mele-Hubbard model with third-neighbor hopping, an interacting two-dimensional model with a topological phase transition as a function of third-neighbor hopping, by means of the determinant projector quantum Monte Carlo method. This technique is essentially numerically exact on models without a fermion sign problem, such as the one we consider. We determine the interaction dependence of the $Z_2$ topological insulator/trivial insulator phase boundary by calculating the $Z_2$ invariants directly from the single-particle Green's function. The interactions push the phase boundary to larger values of third-neighbor hopping, thus, stabilizing the topological phase. The observation of boundary shifting entirely stems from quantum fluctuations. We also identify qualitative features of the single-particle Green's function which are computationally useful in numerical searches for topological phase transitions without the need to compute the full topological invariant.

Figure 1

(a) The first Brillouin zone of the honeycomb lattice. For $t_{3N}=0$, the Dirac points are located at $K_{1,2}=(\pm \frac{4\pi }{3\sqrt{3}a},0)$ labeled by open and solid circles, respectively. The time-reversal invariant momentum (TRIM) points labeled by the green dots are $\Gamma =(0,0),M_{1,2}=(\pm \frac{\pi }{\sqrt{3}a},\frac{\pi }{3a})$, and $M_3=(0,\frac{2\pi }{3a})$. (b) The noninteracting band structure of the generalized KMH model Eq. (1) at $t_{3N}=\frac{1}{3}t$ (here, using ${\lambda }_{SO}=0.3t$). Note that, at the critical point separating the trivial insulator from the TI, the Dirac cones shift to the TRIM points $M_{1,2,3}$, instead, at $K_{1,2}$.

Figure 2

(a)–(c) $Z_2$ invariant at $U/t=2–4$ vs $t_{3N}$. The spin-orbital coupling is ${\lambda }_{SO}=0.4t$. The black squares show the $Z_2$ invariant given by the tight-binding calculations with $200\times 200$. The red circle indicates the $Z_2$ invariant calculated by QMC simulations with $6\times 6$ at $U=0$. The blue solid triangles depict the $Z_2$ invariant of the KMH model at $U\ne 0$. (d)–(f) show the proportional coefficient ${\alpha }_{\mathbf{k}}$ determined by the relation: $G_{\sigma }({\mathbf{k}}_i,0)={\alpha }_{{\mathbf{k}}_i}{\sigma }^x$ from QMC simulations vs $t_{3N}$. All the open symbols indicate noninteracting cases, i.e., $U=0$. The solid symbols denote interacting cases.

Figure 3

The comparison of the single-particle Green's function coefficients ${\alpha }_{M_{1,2}}$ as a function of $t_{3N}$ on $6\times 6$ (open symbols) and $12\times 12$ (solid symbols) for (a) ${\lambda }_{SO}=0.4t$ and $U=4t$ (b) ${\lambda }_{SO}=t$ and $U=6t$. The insets indicate the comparison of the $Z_2$ invariants vs $t_{3N}$ on the $6\times 6$ and $12\times 12$ clusters with the same parameters.

Figure 4

Single-particle excitation gap ${\Delta }_c$ for different values of interaction: $U/t=0–4$ with ${\lambda }_{SO}=0.4t$ on (a) $6\times 6$ and (b) $12\times 12$ clusters. For $U\ne 0$, the single-particle gap remains open across the topological phase transition in contrast to the behavior in a noninteracting system.