Weak Mott insulators on the triangular lattice: Possibility of a gapless nematic quantum spin liquid

We study the energetics of Gutzwiller projected BCS states of various symmetries for the triangular lattice antiferromagnet with a four-particle ring exchange using variational Monte Carlo methods. In a range of parameters the energetically favored state is found to be a projected $d_{x^2-y^2}$ paired state which breaks lattice rotational symmetry. We show that the properties of this nematic or orientationally ordered paired spin-liquid state as a function of temperature and pressure can account for many of the experiments on organic materials. We also study the ring-exchange model with ferromagnetic Heisenberg exchange and find that among the studied ansätze, a projected $f$-wave state is the most favorable.

Figure 1

Angular dependence of superconducting order parameter for the $d_{x^2-y^2}$, $d_{x^2-y^2}+i{d}_{xy}$, and $f_{x^3-3x{y}^2}$ states.

Figure 2

(Color online) (a) Difference between $E_{\text{PBCS}}$ and $E_{\text{PFL}}$ for various paired states in the units of $J_2$. The calculations are done on a $10\times 11$ lattice with antiperiodic boundary conditions using the standard METROPOLIS Monte Carlo (Ref. 21) with ${10}^5$ sweeps. (b) Gap parameter ${\Delta }_0$ for various paired states in units of $t$.

Figure 3

(a) Difference between $E_{Pf\text{−}wave}$ and $E_{\text{PFL}}$ as a function of $J_4/J_2$ for ferromagnetic $J_2$. Both projected nodal $d$ and $d+id$ have optimal value of ${\Delta }_0=0$ for this sign of $J_2$ and hence are not favorable compared to PFL. (b) Gap parameter ${\Delta }_0$ for projected nodal $f$ wave as a function of $J_4/J_2$.

Figure 4

Schematic sketch of the expected specific-heat curve. The left panel shows the mean-field behavior. The full line is in the clean limit and the dashed line the modification due to impurity scattering. The right panel shows the expected behavior when gauge fluctuations beyond mean field are included.

Figure 5

Schematic pressure-temperature phase diagram for the nodal $d$-wave state. Figure (a)/(b) corresponds to the case when the pair order is killed before/after the Mott transition (denoted by “$\times$”). Phase I corresponds to the finite-temperature threefold symmetry-breaking nematic insulator. The details of the regions II and ${\text{II}}^{\prime }$ are determined by the nature of the Mott transition and/or finite-$T$ crossover phenomena associated with the FL/PFL. We do not discuss it in this paper (please see Ref. 26 for a theory of region II). The phase III has long-range nematic order and power-law superconducting order while in the region IV both these orders have only power-law correlations. Please see the text for discussion of the associated phase transitions.