Bridging Hubbard model physics and quantum Hall physics in trilayer $\text{graphene}/h-\mathrm{BN}$ moiré superlattice

The moiré superlattice formed by ABC stacked trilayer graphene aligned with a hexagonal boron nitride substrate (TG/h-BN) provides an interesting system where both the bandwidth and the topology can be tuned by an applied perpendicular electric field $D$. Thus the TG/h-BN system can simulate both Hubbard model physics and nearly flat Chern band physics within one sample. We derive lattice models for both signs of $D$ (which controls the band topology) separately through explicit Wannier orbital construction and mapping of Coulomb interaction. When the bands are topologically trivial, we discuss possible candidates for Mott insulators at integer number of holes per site (labeled as ${\nu }_T$). These include both broken symmetry states and quantum spin liquid insulators which may be particularly favorable in the vicinity of the Mott transition. We propose feasible experiments to study carefully the bandwidth tuned and the doping tuned Mott metal-insulator transition at both ${\nu }_T=1$ and ${\nu }_T=2$. We discuss the interesting possibility of probing experimentally a bandwidth (or doping) controlled continuous Mott transition between a Fermi liquid metal and a quantum spin liquid insulator. Finally we also show that the system has a large valley Zeeman coupling to a small out-of-plane magnetic field, which can be used to control the valley degree of freedom.

Figure 1

Illustration of the ABC stacked trilayer graphene/h-BN system. We assume the h-BN layer on top is nearly aligned with the graphene layers. $A$ and $B$ refer to the two sublattices in each of the graphene layers. Due to the large dimerization term ${\gamma }_1\approx 400$ meV, only $A_1$ and $B_3$ should be kept at low energy, forming a two-component spinor. A vertical electric field gives an energy difference ${\mathrm{\Delta }}_V$ for electrons between the top and the bottom graphene layers. The aligned h-BN layer provides a moiré superlattice potential which folds the original large Brillouin zone to a small moiré Brilloiun zone (MBZ).

Figure 2

Magnitude of the nearest neighbor hopping $|t_1|$ and the next-nearest-neighbor hopping $t_2$. $t_2$ has no imaginary part because of the Mirror reflection symmetry. The phase of $t_1$ is shown in Fig. 3. The vertical line labels ${\mathrm{\Delta }}_V=-20$ meV where the bandwidth is equal to the Hubbard $U$: $W\approx U\approx 25$ meV.

Figure 3

The flux $\left|\mathrm{\Phi }\right|$ of each triangle from the nearest neighbor hopping. For each triangle, two valleys experience opposite $\mathrm{\Phi }$. For each valley, $\mathrm{\Phi }$ changes sign under $C_6$ rotation. The vertical line labels ${\mathrm{\Delta }}_V=-20$ meV where the bandwidth is equal to the Hubbard $U$: $W\approx U\approx 25$ meV. For the Mott insulating regime at ${\mathrm{\Delta }}_V<-25$ meV, we expect a large valley contrasting flux $\left|\mathrm{\Phi }\right|\sim 0.5\pi -2\pi$ trhough each triangle. Such a flux breaks $\mathrm{SU}\left(4\right)$ symmetry, which is inherited in the spin model for the Mott insulator through the superexchange term.

Figure 4

Response to out of plane magnetic field $B$ from the valley Zeeman coupling at ${\mathrm{\Delta }}_V=-25$ meV. $\mathrm{\Delta }E={\overline{E}}_{+}-{\overline{E}}_{-}$ is the splitting of the average energy of the valley + and the valley $-$. $\delta W_a$ is the change of the bandwidth for valley $a$. A small magnetic field $B=1$ T split the average energy for two valleys by about 3 meV. Meanwhile the bandwidth of one valley is increased by around 1.5 meV while the bandwidth of the other valley is reduced by around 1.5 meV.

Figure 5

$J_1-J_2$ parameters with ${\mathrm{\Delta }}_V$. We fix $U=25$ meV, $g_1=0.4$ and $g_h=0.008$ in the calculation. The vertical line is the value of ${\mathrm{\Delta }}_V$ for which the bandwidth $W=U$. Deep inside the Mott insulating phase, $J_1$ is ferromagnetic from the Hund's coupling. In the intermediate regime, both $J_1$ and $J_2$ are antiferromagnetic.

Figure 6

Three possible phase diagrams tuned by $t/U$ at ${\nu }_T=2$. In (I), AF Metal means metal coexisting with antiferromagnetic order. AF insulator is a Mott insulator with antiferromagnetic order (The most likely candidate is the ${120}^{\circ }$ valley order). In (II), the Mott insulator may be antiferromagnetic or may be a quantum spin liquid. In (III), the specific quantum spin liquid we consider has a spinon Fermi surface coupled to a $\mathrm{U}\left(1\right)$ gauge field. At $\frac{t}{U}=0$, the ground state is a ferromagnet because of intersite Hund's coupling. We also show the plots of the Fermi surfaces. The Fermi surfaces are calculated at ${\nu }_T=2$ using $t_1,t_2,t_3,t_4$ for ${\mathrm{\Delta }}_V=-20$ meV. For the AF metal, we use the ${120}^{\circ }$ intervalley order with the order parameter $M=2|t_1|$. The Fermi surface area should decrease continuously in the AF metal region as $\frac{M}{|t_1|}$ increases.

Figure 7

The change of Fermi surface area(in units of the area of the MBZ) with order parameter $M$ for the ${120}^{\circ }$ intervalley order: $H_M=-M{\sum }_{\mathbf{x}}{c}_{\mathbf{x}}^{†}(cos(\mathbf{Q}·\mathbf{x}){\tau }_x+sin(\mathbf{Q}·\mathbf{x}){\tau }_y)c_{\mathbf{x}}$ with $\mathbf{Q}=(\frac{4\pi }{3},0)$. We use $t_1=2.14e^{i0.141\pi }$ meV and $t_2=-1.372$ meV for ${\mathrm{\Delta }}_V=-20$ meV. There are several Fermi surfaces and we only count the hole pocket at $\mathrm{\Gamma }$ point. At $M\to 0$, magnetic breakdown effect should give a quantum oscillation frequency corresponding to the original Fermi surface area equal to 0.5, which is not captured by our calculation here. After adding a nonzero $t_2$, Fermi surfaces can not be fully gapped out until $M=8|t_1|$.

Figure 8

Density of state at ${\mathrm{\Delta }}_V=-25$ meV. Two vertical lines correspond to ${\nu }_T=1$ and ${\nu }_T=2$. The Van Hove singularity is away from both ${\nu }_T=1$ and ${\nu }_T=2$. This is true for other values of $D$ in the region $-40<{\mathrm{\Delta }}_V<-5$ meV. The closest distance to ${\nu }_T=1$ for the Van-Hove singularity is still at least $10\%$ doping away. The Van-Hove singularity is associated with a Lifshitz transition of the Fermi surfaces (see Appendix pp1). At exactly ${\nu }_T=1\text{and}2$, there is no obvious instability for the Fermi surfaces.

Figure 9

Illustration of bandwidth-controlled metal-insulator transition (BMIT) and doping-controlled metal-insulator transition (DMIT). The shaded region is the Mott insulator.

Figure 10

Response to out of plane magnetic field $B$ from the valley Zeeman coupling at ${\mathrm{\Delta }}_V=25$ meV. $\mathrm{\Delta }E={\overline{E}}_{+}-{\overline{E}}_{-}$ is the splitting of the average energy of the valley + and the valley $-$. $\delta W_a$ is the change of the bandwidth for valley $a$.

Figure 11

The dependence of the bandwidth $W$ and the band gap ${\mathrm{\Delta }}_2$ on the applied vertical voltage difference $D$. ${\mathrm{\Delta }}_2$ is the minimal gap between the valence band and the band below. $W\text{and}{\mathrm{\Delta }}_2$ are all in units of meV. The band gap ${\mathrm{\Delta }}_1$ between the conduction band and the valence band is almost equal to $|{\mathrm{\Delta }}_V|$ and becomes larger than the bandwidth after $|{\mathrm{\Delta }}_V|>30$ meV.

Figure 12

Band structures of the valence bands in the hole picture for ${\mathrm{\Delta }}_V=-25$ and 25 meV in $ab$ out-of-plane magnetic field $B$. $K^{\prime }=(0,\frac{4\pi }{3a_M})$. $K^{\prime \prime }=(\frac{2\pi }{\sqrt{3}{a}_M},\frac{2\pi }{3a_M})$, and $K=(0,-\frac{4\pi }{3a_M})$ are equivalent in the MBZ. Two horizontal lines are the chemical potential for ${\nu }_T=1$ and ${\nu }_T=2$. For ${\mathrm{\Delta }}_V<0$, out-of-plane magnetic field split the energies of two valleys. It also increases the bandwidth of one valley while reducing the bandwidth of the other valley. For ${\mathrm{\Delta }}_V>0$, out-of-plane magnetic field increase the bandwidth of one valley while decrease the bandwidth of the other valley.

Figure 13

Fermi surfaces at ${\nu }_T=1$ and ${\nu }_T=2$. Red and blue lines denotes the Fermi surface contours for the two different valleys. For ${\nu }_T=1$, there are three separate Fermi surfaces related by the $C_3$ symmetry. When increasing ${\nu }_T$, there is a Lifshitz transition to an annulus-shape Fermi sea. At ${\nu }_T=2$, the Fermi surface is simply a circle for each valley.