Exotic quantum phase transitions of strongly interacting topological insulators

Using determinant quantum Monte Carlo simulations, we demonstrate that an extended Hubbard model on a bilayer honeycomb lattice has two novel quantum phase transitions. The first is a quantum phase transition between the weakly interacting gapless Dirac fermion phase and a strongly interacting fully gapped and symmetric trivial phase, which cannot be described by the standard Gross-Neveu model. The second is a quantum critical point between a quantum spin Hall insulator with spin $S^z$ conservation and the previously mentioned strongly interacting fully gapped phase. At the latter quantum critical point the single-particle excitations remain gapped, while spin and charge gaps both close. We argue that the first quantum phase transition is related to the ${\mathbb{Z}}_{16}$ classification of the topological superconductor ${}^3\mathrm{He}\text{−}B$ phase with interactions, while the second quantum phase transition is a topological phase transition described by a bosonic $O\left(4\right)$ nonlinear sigma model field theory with a $\Theta$ term.

Figure 1

The bilayer honeycomb lattice. In each layer, $t$ and $\lambda$ are the nearest- and next-nearest-neighbor hopping. The Hubbard interaction $U$ acts on each site, and the Heisenberg interaction $J$ acts across the layers.

Figure 2

A schematic phase diagram of the bilayer honeycomb model. The red line is the phase boundary between the two QSH phases of opposite spin Hall conductivity, where both the single-particle and the spin/charge gaps are closed. The blue line is the phase boundary between the QSH phase $\Theta =\pm 2\pi$ and the trivial gapped phase $\Theta =0$, where the single-particle gap remains open but the spin/charge gaps are closed. $U_c$ is the tricritical point, above which the topological number defined in Eq. (6) changes inside the trivial phase (without gap closing) through the dashed line; also see Fig. 3.

Figure 3

The topological number defined in Eq. (6) as a function of $\lambda$ for both models at $U=2$. The topological number was calculated at the dots using determinant QMC data via the methods discussed in Appendix pp6. This demonstrates that this topological number Eq. (6) is nonzero even in the strongly interacting trivial phase.

Figure 4

Single particle and spin gap for the $1d$ coupled chain model with $J/U=2$. (a) When $\lambda =0$, the system is gapped out immediately by an infinitesimal interaction with a gap of the form $e^{a-b/U}$ for small $U$ (dotted black line with $a=2.60$ and $b=2.65$). (b) When $\lambda =0.25$, there are no phase transitions when $\lambda \ne 0$ and $U>0$.

Figure 5

Single-particle gap, spin gap (gap for spin-1 excitation), and charge gap (gap for charge-2 excitation) on the bilayer honeycomb lattice with $J/U=2$. (a) When $\lambda =0$, there is a single continuous phase transition from a semimetal to a trivial insulator at $U_c\sim 1$, whose field theory also describes the phase transition of the boundary of 16 copies of the ${}^3\mathrm{He}\text{−}B$ phase. (b) When $\lambda =0.25$, only the spin and charge gap close at the continuous phase transition from an SPT to a trivial insulator (which is at $U_c\sim 1.5$ for $\lambda =0.25$). We propose that this phase transition is described by a bosonic $O\left(4\right)$ nonlinear sigma model field theory with a $\Theta$ term [Eq. (5)]. These gaps are calculated as explained in Appendix pp5. This involves calculating gaps in finite systems of sizes up to $9\times 9$ unit cells (with 4 sites each) and extrapolating to the infinite-size limit. Error bars on all figures denote one standard deviation (i.e., $\approx 68\%$ confidence).

Figure 6

Green's function $G(i\omega ,K)$ as a function of frequency at the $K$ point with $\lambda =0$ and $J/U=2$ on the bilayer honeycomb lattice for various system sizes. [The largest eigenvalue of $G(i\omega =0,K)$ is shown.] (a) In the free fermion limit when $U\ll U_c\sim 1.5$, the Green's function shows a pole at zero frequency: $G(i\omega ,K)\simeq 1/\left(i\omega \right)$ [Eq. (B2)] (dotted black line). (b) In the strong interacting limit when $U\gg U_c\sim 1.5$, the Green's function follows the behavior of $G(i\omega ,K)\simeq \left(i\omega \right)/[{\left(i\omega \right)}^2-{\Delta }^2]$ [as calculated in Eq. (B3)] (dotted black line) where $\Delta$ is the quasiparticle gap. Please note that here $\mathrm{Im}G$ is the imaginary part of the imaginary-time Green's function, which is very different from the spectral function.

Figure 7

Zero-frequency Green's function $G(i\omega =0,K)$ at the $K$ point with $J/U=2$ on the bilayer honeycomb lattice for various system sizes. [The largest eigenvalue of $G(i\omega =0,K)$ is shown.] In the free fermion limit when $U\ll U_c\sim 1.5$, the Green's function decays as $G(i\omega =0,K)\simeq 1/3\sqrt{3}\lambda$ [cf. Eq. (B2)] (dotted black line).

Figure 8

The exponential decay in imaginary time of correlation functions (red line) for various order parameters (Table 1) on a honeycomb lattice of dimension $3\times 3$ with $U=1.4375$, which is nearly at the critical point. The shaded red region denotes statistical errors. The thick green line indicates the fit to $e^{-\tau \Delta +c}$ while the two thin green lines denote the uncertainty of the fit. The fit was performed in the region between the vertical orange lines. The negative of the slope of the fit is the energy gap for the finite-size system, which is used to make Fig. 9.

Figure 9

We extrapolate the gaps associated with a single particle (a), spin (b), and charge (c) (Table 1) from a system of finite spatial size to one of infinite size. Extrapolations are shown for $\lambda =0.25$ and interaction strengths below $\left(U=1\right)$, near $\left(U=1.4375\right)$, and above $\left(U=2\right)$ the gapless critical point at $U\sim 1.5$. The results of these extrapolations are used to make Fig. 5 ($\lambda =0.25$ and $J/U=2$).