Quantum phase transitions in the Kane-Mele-Hubbard model

We study the two-dimensional Kane-Mele-Hubbard model at half filling by means of quantum Monte Carlo simulations. We present a refined phase boundary for the quantum spin liquid. The topological insulator at finite Hubbard interaction strength is adiabatically connected to the groundstate of the Kane-Mele model. In the presence of spin-orbit coupling, magnetic order at large Hubbard $U$ is restricted to the transverse direction. The transition from the topological band insulator to the antiferromagnetic Mott insulator is in the universality class of the three-dimensional XY model. The numerical data suggest that the spin liquid to topological insulator and spin liquid to Mott insulator transitions are both continuous.

Figure 1

Ground-state phase diagram of the Kane-Mele-Hubbard model as obtained from QMC simulations. The four phases are a $Z_2$ topological band insulator (TBI) with nonzero single-particle (spin) gap ${\Delta }_{\mathrm{sp}}>0$ (${\Delta }_{\mathrm{s}}>0$), a semimetal (SM, ${\Delta }_{\mathrm{sp}}={\Delta }_{\mathrm{s}}=0$) existing at $\lambda =0$, a quantum spin liquid (QSL, ${\Delta }_{\mathrm{sp}}>0$, ${\Delta }_{\mathrm{s}}>0$), and an antiferromagnetic Mott insulator (AFMI, ${\Delta }_{\mathrm{sp}}>0$, ${\Delta }_{\mathrm{s}}=0$). Magnetic order in the $z$ direction exists in the AFMI at $\lambda =0$ but can be excluded for $\lambda /t\ge 0.002$ and all values of $U/t$ shown. Lines are quadratic fits to the QMC data points.

Figure 2

(a) Single-particle gap ${\Delta }_{\mathrm{sp}}$, $2{\Delta }_{\mathrm{sp}}$, and spin gap ${\Delta }_{\mathrm{s}}$ as a function of $\lambda$ at $U/t=2$, across the SM–TBI transition. The QMC results are obtained from finite-size extrapolation using the fitting function Eq. (21) at $\lambda =0$ and Eq. (20) for $\lambda >0$. The numerical results shown as symbols agree well with the corresponding gaps of the KM model ($U=0$),[5] ${\Delta }_{\mathrm{sp}}=3\sqrt{3}\lambda$, and ${\Delta }_{\mathrm{s}}=2{\Delta }_{\mathrm{sp}}$, revealing the adiabatic connection between the TBI at $U=0$ and at $U>0$. (b) Finite-size scaling of the single-particle gap at selected values of $\lambda /t$. (c) Finite-size scaling of the spin gap; for $\lambda >0$, we neglect the $L=3$ results in the extrapolation.

Figure 3

Finite-size scaling of the rescaled magnetic structure factor $S_{\mathrm{AF}}^{xy}/N$ defined in Eq. (15) at $\lambda /t=0.1$ for different values of $U/t$, across the TBI–AFMI transition. The curves (representing polynomial fits) extrapolate to zero for $U/t<4.9$ and to a finite value for $U/t\ge 5.0$, giving the critical value $U_{\mathrm{c}}/t=4.95\left(5\right)$. A more accurate estimate $U_{\mathrm{c}}/t=4.96\left(4\right)$ is obtained from Fig. 5a. The inset shows the order parameter $m^{xy}$ as obtained from extrapolation to the thermodynamic limit.

Figure 4

(a) Single-particle gap ${\Delta }_{\mathrm{sp}}$ and spin gap ${\Delta }_{\mathrm{s}}$ as a function of $U$ at $\lambda /t=0.1$. The values shown were obtained from an extrapolation to the thermodynamic limit. The dip in ${\Delta }_{\mathrm{sp}}$ and the closing of ${\Delta }_{\mathrm{s}}$ are consistent with $U_{\mathrm{c}}/t=4.95\left(5\right)$. The inset shows the mean-field results for the single-particle gap and the magnetic order parameter. (b) Energy derivative with respect to $U$ [Eq. (16)] across the TBI–AFMI transition at $\lambda /t=0.1$ [$U_{\mathrm{c}}/t=4.95\left(5\right)$].

Figure 5

Rescaled transverse magnetic structure factor $S_{\mathrm{AF}}^{xy}/N$ defined in Eq. (15) as a function of $U$ at $\lambda /t=0.1$, for different lattice sizes $L$. Assuming the scaling form Eq. (17), (a) shows $L^{2\beta /\nu }{S}_{\mathrm{AF}}^{xy}/N$. The intersection of curves for different system sizes yields $U_{\mathrm{c}}/t=4.96\left(4\right)$ for the critical point. (b) The scaling collapse obtained by plotting $L^{2\beta /\nu }{S}_{\mathrm{AF}}^{xy}/N$ as a function of $L^{1/\nu }(U-U_{\mathrm{c}})/U_{\mathrm{c}}$. The QMC data are fully consistent with the critical exponents $z=1$, $\nu =0.6717\left(1\right)$, and $\beta =0.3486\left(1\right)$ of the 3D XY model.[33]

Figure 6

Spin gap ${\Delta }_{\mathrm{s}}$ as a function of $U$ at $\lambda /t=0.1$, for different lattice sizes $L$. Given the scaling form Eq. (18), (a) shows $L^z{\Delta }_{\mathrm{s}}$. The intersection of curves for different $L$ gives $U_{\mathrm{c}}/t=4.96\left(4\right)$, consistent with Fig. 5a. The inset shows the finite-size scaling of ${\Delta }_{\mathrm{s}}$. (b) Scaling collapse obtained by plotting $L^z{\Delta }_{\mathrm{s}}$ as a function of $L^{1/\nu }(U-U_{\mathrm{c}})/U_{\mathrm{c}}$. The QMC data are consistent with the 3D XY exponents $z=1$, $\nu =0.6717\left(1\right)$, and $\beta =0.3486\left(1\right)$.[33]

Figure 7

(a) Finite-size scaling of the rescaled magnetic structure factor $S_{\mathrm{AF}}^{xy}/N$ defined in Eq. (15) at $\lambda /t=0.0125$ for different values of $U/t$, across the QSL–AFMI transition. The data suggest a magnetic transition at $U_{\mathrm{c}}/t=4.3\left(2\right)$. Lines correspond to polynomial fits. (b) Same as in (a) but showing the longitudinal structure factor $S_{\mathrm{AF}}^{zz}$ defined in Eq. (19). In contrast to (a), there is no long-range order over the range of $U/t$ values considered.

Figure 8

Energy derivative with respect to $U$ [Eq. (16)] across the QSL–AFMI transition at $\lambda /t=0.0125$ [$U_{\mathrm{c}}/t=4.3\left(2\right)$], and at $\lambda /t=0.04$ [$U_{\mathrm{c}}/t=4.3\left(2\right)$], close to the multicritical point. There are no signs of a first-order transition.

Figure 9

(a) Single-particle gap ${\Delta }_{\mathrm{sp}}$ and (b) spin gap ${\Delta }_{\mathrm{s}}$ and inverse spin correlation length $1/{\xi }_{\mathrm{s}}^{\prime }$ as a function of $\lambda$ at $U/t=4$, across the QSL–TBI transition. ${\Delta }_{\mathrm{sp}}$ is obtained from finite-size scaling, using Eq. (20) for $\lambda /t\ge 0.045$ and Eq. (21) for $\lambda /t<0.045$. The cusp defines the critical coupling ${\lambda }_{\mathrm{c}}/t=0.030\left(1\right)$. The inset in (a) shows the scaling for selected values of $\lambda /t$. ${\Delta }_{\mathrm{s}}$ is obtained from finite-size scaling using Eq. (20); see (c). ${\xi }_{\mathrm{s}}^{\prime }$ is extracted from fits to the spin-spin correlation function defined in Eq. (22) at $r=L/2$; see inset in (b).

Figure 10

Energy derivative with respect to $\lambda$ [Eq. (23)], across the QSL–TBI transition at $U/t=4$. There is no sign of a discontinuity at ${\lambda }_{\mathrm{c}}/t=0.030\left(1\right)$.