Helical spin liquid in a triangular XXZ magnet from Chern-Simons theory

We propose a finite-temperature phase diagram for the 2D spin-$1/2$ $J_1-J_2$ $XXZ$ antiferromagnet on the triangular lattice. Our analysis, based on a composite fermion representation, yields several phases. This includes a zero-temperature helical spin liquid with $N=6$ {\it anisotropic} Dirac cones, and with nonzero vector chirality implying a broken $\mathbb{Z}_2$ symmetry. It is terminated at $T=0$ by a continuous quantum phase transition to $120^\circ$ ordered state around $J_2/J_1\approx0.089$ in the XX limit; these phases share a double degeneracy, which persists to finite $T$ above the helical spin liquid. By contrast, at $J_2/J_1 \simeq 0.116$, the transition into a stripe phase appears as first order. We further discuss experimental and numerical consequences of the helical order and the anisotropic nature of the Dirac dispersion.

Introduction. Two-dimensional s = 1/2 magnets with frustrated interactions attract a great deal of interest because of their potential to host unconventional states of quantum matter such as spin liquids (SL) [1][2][3][4][5][6][7][8][9] Quantum SL are long range entangled states that give rise to emergent gauge fields and represent deconfined phases, where the quasi-particles exhibit fractional quantum numbers. They do not break rotational symmetry, thus excluding orientational long range ordering. Traditionally, the triangular lattice has been regarded as a promising ground for realization of a SL [10][11][12][13][14][15]. In this setting, the frustrated spin-1/2 J 1 − J 2 XXZ antiferromagnet on a triangular lattice is one such candidates for a SL ground state in a parameter window around J 2 /J 1 ∼ 0.1. The Hamiltonian of the model is given bŷ where l = 1, 2 and a parameter g measures the anisotropy of the interactions. Vectors µ 1 ν = e ν and µ 2 ν = a ν , ν = 1, 2, 3, point to nearest neighbor (NN) and next nearest neighbor (NNN) sites on the triangular lattice, respectively. The spin-orbit coupled version of the model is believed to be related to the triangular lattice antiferromagnet YbMgGaO 4 [16][17][18][19][20][21].
The Heisenberg model, g = 1, has been studied numerically using variational Monte-Carlo [22][23][24][25] and the density matrix renormalization group (DMRG) [26][27][28][29]. The variational Monte-Carlo study of Ref. [24] explores the phase diagram of the model Eq. (1) ranging from the XX limit, g = 0, all the way to the Heisenberg limit with g = 1. The nature of the SL was identified with the U (1) Dirac gauge theory which emerges in the Heisenberg limit in an approximate parameter interval 0.08 t 0.2, t = J 2 /J 1 [23,25]. The interval where the SL is realized appears to be narrower in the XX limit [24]. In this letter we carry out an analysis based on a composite Fermion representation. An advantage of the fermion representation is that it can be used to effectively describe both the ordered phase as a CS superconductor [30], and the spin-liquid, where the fermions are "deconfined." This leads us to propose that the Dirac SL, which restores the rotational O(2) symmetry, is stabilized in the XXZ model in a narrow interval of parameter t = J 2 /J 1 , eg. 0.089 t 0.116 in XX limit. The nature of this SL appears to be different from the Heisenberg limit in a crucial way: in the XXZ model arXiv:1911.02932v1 [cond-mat.str-el] 7 Nov 2019 it exhibits spontaneous breaking of the Z 2 symmetry, inherent to the 120 • antiferromagnetic ordering of the XX model at J 2 = 0. We thus predict a SL with long range Z 2 order and the vector chirality playing the role of the order parameter which distinguishes between two degenerate SL ground states. The other central finding is that such a SL is described in terms of N = 6 copies of Dirac fermions. Each Dirac cone is predicted to exhibit a uniaxial anisotropy, although the complete spectrum preserves the C 3 symmetry of the lattice. A finite vector chirality and the anisotropy of individual Dirac cones are the main results of this letter, which may be checked within DMRG, tensor network, or variational Monte-Carlo approaches. They may be also observed in spin resolved neutron scattering.
Furthermore, our theory predicts a rich finitetemperature phase diagram, Fig. 1. As in the classical XX model [31][32][33], a BKT transition takes place first into a helical phase with restored O(2), but broken Z 2 symmetries. At yet higher temperature, there is an Ising-like transition to a disordered paramagnet. Finally, we argue that the T = 0 transition from 120 • ordered state to the helical SL is continuous, while the one into the stripe phase is first order.
We treat these transitions by first developing a theory of the 120 • state via Chern-Simons (CS) superconductivity [30], considering the stability of the superconducting solution upon increasing t. The superconducting order breaks down at t ∼ 0.089 for g = 0, signaling an emergence of the Dirac SL state with broken Z 2 symmetry. Next, we identify a CS superconductor describing the collinear stripe phase, energetically favorable beyond t ∼ 0.116.
The Hamiltonian (1) can be regarded as a model of hard-core bosons hopping on a triangular lattice with NN amplitude J 1 and NNN amplitude J 2 . At small J 2 (small t < 1/8), the single boson dispersion exhibits two degenerate minima located at the K and K points of the Brillouin zone (BZ), Fig. 2a. This implies that noninteracting bosons can condense to any superposition of these two states, however the hard-core interactions prevent forming a density modulation and enforce condensation into one of these two points. This leads to the doubly degenerate ground states, identified with the planar 120 • Néel configurations of spins with two helicities, Figs. 2b,c. At t = 1/9 the single particle dispersion acquires an additional minimum at the M point, midway between K and K , while at t = 1/8 the dispersion is triply degenerate, Figs. 2d and 2e. At t > 1/8 the global minimum is at M , signaling semiclassically a first order transition [34,35] into a state with the collinear stripe order shown in Fig. 2g.
Emergence of anisotropic Dirac fermions. This picture is severely modified by quantum fluctuations. We account for these by reformulating the model (1) as a theory of spinless lattice fermions coupled to a CS gauge field. The fermionization automatically takes care of the hard-core condition. The spin operators may be represented as S ± r = exp ie r =r arg[r − r ]n r f ± r , where e = 2l + 1 is an odd integer representing the CS charge, f ± r are creation/annihilation operators of canonical spinless fermions, n r = f † r f r = S + r S − r is the particle number operator, and summation runs over all lattice sites. The XX part of the Hamiltonian (1) acquires the form: where Λ r1,r2 = r [arg(r 1 − r) − arg(r 2 − r)] n r is a gauge field associated with the NN and NNN links on the triangular lattice. It introduces CS magnetic flux threading the unit cell of the triangular lattice: Φ r = Λ r,r+e1 +Λ r,r+e2 +Λ r+e1,r+e1+e2 +Λ r+e2,r+e1+e2 = 2πn r , which is the lattice analog of Φ r = curl Λ. The Hamiltonian (1) thus can be rewritten in terms of fermions [30,[36][37][38][39][40], f r , coupled to the U (1) CS gauge field [37].
To illustrate how this U (1) field affects the fermion dynamics, we analyze the XX limit of the Hamiltonian (1). In the absence of a net magnetization the CS fermion state is half-filled, n = 1/2. This implies that the CS phases create 2iπ n → iπ flux, threading the unit cell. The double degeneracy (two helicities) of the 120 • state is reflected in the staggered π flux patterns, Figs. 3a,b, for which there are two inequivalent choices, distinguished by the sign of the z-component of the vector chirality, defined on a triangular plaquette as [41,42] where . . . stands for the quantum expectation value and i, j = x, y. Importantly, this Z 2 order parameter, reflecting the two different π flux patterns, persists in the SL phase, again implying a double degeneracy.
FIG. 3: (Color online) a) and b) Distinct π flux configurations of CS fermionization, corresponding to two helicities of the 120 • order. c) π flux configuration corresponding to the stripe order.
Consider now the fermionized Hamiltonian for J 2 > 0. The NNN bonds form three disjoint large triangular lattices, labelled as τ = 1, 2, 3. At half-filling, fermions hopping along these bonds still accumulate a π flux through a large rhomboidal cell composed of NNN links. The CS transformation unambiguously identifies the π-flux configuration depicted in Fig. 4a for one of the three sublattices, τ = 1. Flux patterns of the two other NNN sublattices are obtained by rotation of Fig. 4a by ±2π/3. Such an arrangement of π fluxes on three NNN triangular lattices preserves C 3 symmetry. However, it further violates the translational invariance in three lattice spacings and, thus the unit cell becomes 6 times larger than the original one. Correspondingly, there is a 6 component Fermi field, f τ k,α , where α = 1, 2 and τ = 1, 2, 3, in a 6 times reduced BZ, leading to a 6 × 6 Hamiltonian [43].
and its time-reversed partners H (Pτ ) 0 withf τ k,α →f τ k,α , and v = J 1 a, a being the lattice constant. We use: p ν = k · e ν and q ν = k · a ν , ν = 1, 2, 3. Hereafter we work with dimensionless quantities measuring energy in units of J 1 and momentum in units of 1/a. The low-energy fermion operators,f τ k,α andf τ k,α have momenta measured from P andP points, respectively. The C 3 invariance is ensured by 2π/3 rotations of the lattice accompanied with cyclic transformation, τ → τ + 1, of the fermion copy. The Hamiltonian (4) leads to the anisotropic spectrum The 120 • state with the other helicity corresponds to Note that the ground state spontaneously chooses one of the two.
Upon a gauge transformation, the CS phases in Eq. (2) may be rewritten as covariant derivatives, leading to the substitution k → k − eA r in Eq. (4). Here A r in the kinetic term reflects fluctuations of the CS phases from 0 or π per plaquette and is bilinear in fermion operators. It thus generates a two particle interaction vertex [30,36] is determined by the Fourier image A k = k/|k| 2 of the vector potential of the vortex gauge field A r defined above.
Chern-Simons superconductor description of the 120 • state.-The CS interaction (6) leads to the Cooper pairing of fermions residing near the P andP points and may result in a broken U(1) superconducting phase. In terms of the original spins the latter corresponds to a broken O(2) 120 • antiferromagnet. Upon increasing t, the fermion dispersion becomes more anisotropic, weakening the Cooper pairing (which operates only within time-reversal pairs with the same NNN sub-lattice index τ , Eq. (6), which has a non-collinear anisotropy orientation in P andP points, Fig. 4b). This leads to an eventual breakdown of the CS superconductivity at t ≈ 0.089.
To describe this physics we employ the standard BCS treatment with the superconducting order parameter ∆ αα The order parameter is quadratically coupled to the fermions as: , leading to self-consistency conditions. Following Ref. [30], one expects p ± ip symmetry of ∆ αα k and thus looks for the solution in the form where η µν is defined as η 33 = 1, η ij = ij , η 3i = 0, i, j = 1, 2. The corresponding self-consistency equations in terms of the scalar order parameters ∆ 0,k and ∆ 3,k are given in the supplementary material [43]. At t = 0  they exhibit a non-trivial solution [30] for CS charge e = 3. For t > 0 the anisotropy of the Dirac spectrum (5), Fig. 4b, suppresses the order parameter. Fig. 5a shows the corresponding gap in the fermionic spectrum, ∆ 120 • , obtained through a numerical solution of the selfconsistency equations [43]. The gap and the U(1) broken state collapse at t = 0.089, indicating the absence of the planar long range order at larger t. Notice that the Z 2 symmetry breaking, associated with the choice of CS flux pattern, Fig. 3a,b, remains intact across this transition.
Helical SL phase.-For t > 0.089, in the CS superconductivity associated with one of the two 120 • states, one is left thus with a gapless state with an unbroken U(1) symmetry and excitations described in terms of N = 6 copies of Dirac fermions with the anisotropic dispersion. This is the SL ground state, doubly degenerate due to the presence of the the long-range Z 2 order. The latter may be detected by a finite value of the vector chirality, Eq. (3). To derive an effective low-energy field theory in this regime one should integrate out fermionic de-grees of freedom with momenta away from the two Dirac points. This way one obtains a stable [44] 2+1 dimensional Maxwell electrodynamics coupled to N = 6 copies of the anisotropic Dirac fermions [43].
An external magnetic field in the z direction, H h = h r S z r , leads to a deviation from half filling and thus to a non-zero chemical potential of the Dirac fermions. Each additional fermion comes with an extra flux quantum of the gauge field. It results in Landau quantization of the fermionic energies with the fully filled levels. Thus the excitation spectrum of the SL in a magnetic field is gapped with the gap proportional to |h|.
CS superconductor description of the stripe phase. -Consider an alternative choice of CS flux pattern, depicted in Fig. 3c. This choice breaks the C 3 symmetry and selects a preferred direction along the lattice. The reduced BZ can be chosen to be the same as in Fig. 4b, but the Dirac points are Q ± = (± arccos(t), 0). The superconducting solution [43] exists for t 0.113. However its energy is smaller than that of SL only at t 0.116, Fig. 5b, suggesting a first order transition. The corresponding U(1) broken state is the stripe phase, Fig. 2g, which also breaks C 3 lattice symmetry. As a result, the helical SL state appears stabilized in the narrow interval 0.089 t 0.116.
Discussion and estimates.-We formulated a framework where the ordered states of the XXZ magnet are treated as superconducting states of spinless CS fermions. The SL emerges in the parameter window where such superconducting states cannot be established. It would be desirable to test these predictions with, say, DMRG simuations, although the large number of Dirac points involved may make this technically challenging.
We have mostly focused on the XX case, g = 0, while a finite g leads to an additional fermionic interaction vertex, which modifies the self-consistency equations. This leads in turn to a weak g-dependence of the critical values of t. Since the model does not have SU(2) symmetry, the low-energy fermion excitations may be considered spinless, and as such, the SL is outside the projective symmetry group classification of SL's based on Schwinger boson and Abrikosov fermion representation of spins [14,45,46]. We encourage neutron scattering experiments for observation of the proposed spin liquid state. Here the spin magnetization distribution and spin-spin correlators are expected not to exhibit Bragg peaks. Moreover, the anisotropy of the dynamical structure factor near Dirac points may serve as a signal of the anisotropic Dirac dispersion. The information about the vector chiral order of the spin-liquid may be revealed in nuclear-magnetic interferences in the chiral magnetic scattering [47][48][49] of an initially unpolarized neutron beam.
Supplementary material for "Nematic Dirac Spin Liquid in an XXZ Magnet on the Triangular Lattice"

Hamiltonian
The CS transformation [S1] implies a staggered π flux distribution within the NN and NNN triangular sub-lattices. The flux distribution in the NNN triangular sublattices with τ = 2, 3 is obtained from the lattice corresponding to τ = 1 upon rotation by ±2π/3, respectively. Such a phase distribution breaks translational invariance on three lattice steps reducing the Brillouin zone of the NN triangular lattice 6 times.
As a result of the staggered π-flux threading of every other triangle in the unit cell (including both tangles composed on NN and NNN bonds), the single-fermion dispersion on a triangular lattice will acquire a Dirac form around the following points of the first Brillouin zone: P τ = (π/6 + 2πτ /3, π/2 √ 3) andP τ = (−π/6 + 2πτ /3, π/2 √ 3), τ = 1, 2, 3, giving rise to six components of the Fermi field. These Dirac points form a triangular lattice in momentum space (see 4b of the main text), while the reduced BZ is a rhombus that includes only one pair of points (P τ ,P τ ).
The double degeneracy of the planar 120 • state is linked to the interchange of fluxes π ↔ 0 threading each triangular face of the unit cell. We note that the chiralities of the Hamiltonian expanded around P τ andP τ are opposite. Similarly, if one identifies the refletion of P τ andP τ with respect to the k x -axis with P τ andP τ , then the chiralities of the corresponding Hamiltonians (expanded around P τ andP τ ) will also be opposite to each other. This implies that the single particle states at t → 0, in close vicinities of these Dirac points, are given byū and u τ (k) =ū τ (k) = [u τ (k)] * . These states define Berry connections [S3], as , and the Berry phases defined by contours C τ , τ = 1, 2, 3 encircling both, P τ and P τ points, as γ τ = Cτ dk A τ , and γ τ = Cτ dk A τ . The result of integration around each of these Dirac points yields The corresponding Hamiltonian has the form At t = J 2 /J 1 = 0, when we only have a small NN triangular lattice, the spectrum has a simple form. It becomes a spectrum of three Dirac pairs, having zeros at different points.
The terms 2π(τ − 2)/3 in the arguments of the cosine functions appear due to the relative 2π/3 rotations of three NNN sublattices. The ∝ J 2 terms in the Hamiltonian (S1) also have zeros at the same points, therefore common Dirac points are P τ andP τ , τ = 1, 2, 3. The chirality of pointP τ τ = 1, 2, 3, is opposite to that of P τ . The linear expansion of the spectrum around Dirac points gives anisotropic dispersion E 0

Self-consistency equations
Chern-Simons superconductor description of the 120 • phase .
Following the steps outlined in Ref. S2, we obtain the BdG Hamiltonian of the mean-field CS superconductor. In terms of the scalar order parameters ∆ 0,k and ∆ 3,k , the self-consistency conditions take the closed form: where u = (av 0 −(1+t 2 )∆ 0k ) and The order parameters, Eq. (S2), define the four-band Bogolyubov spectra, ±E (a) τ,k , a = ±, of gapped fermions as At t = 0, the self-consistency equations are independent of any continuous parameters (momentum cutoff is defined by the size of the Brillouin zone and is not a model parameter). The existence of the solution within superconducting mean-field approach depends on the interaction strength, and thus on the CS charge, e = 1, 3, 5 . . .. Here we have one continuous parameter, t, and the anisotropy parameter g is set to zero. At t = 0, the only solution that corresponds to e = 1 is the trivial one, where there is no superconducting order. However, if the CS-fermionization is realized with e = 3, a nontrivial solution of gap equations emerges for 0 t 0.089. One can see that Eqs. (S2) coincide with the analogous self-consistency relations of the CS superconductor on the honeycomb lattice at t = 0, first derived in Ref. S2. This indicates the lattice independent 'universal' character of CS superconductivity.
Chern-Simons superconductor description of the stripe phase .
Here we proceed with the fermionic description of the collinear stripe phase. The corresponding π-flux configuration is shown in Fig. 2d of the main text. The BZ is still the same, but Dirac points now are located at Q = (arccos(t), 0) and Q = (− arccos(t), 0). We see that the parity transformations, P x/y , transform one Dirac point to the other, indicating that the ground state does not support the degeneracy of the 120 • ordered state. In the vicinity of these Dirac points the Hamiltonian acquires an especially simple form: with (1 − 2t)(1 + t), and H str,0 (k). The generated interaction vertex, V αα ,ββ q , in this case is similar to the one of the 120 • phase given below Eq. (6), but with B 1 q = −v y A x q , B 2 q = 0, B 3 q = −v x A y q . As in the case of the 120 • ordered state, here we also expect that ∆ αα k has a p ± ip-wave symmetry, and the self-consistency relations are given by Eq.

(S5)
Emergence of the Dirac Spin-Liquid As we see in Fig. 5, massless Dirac fermions emerge in the parameter interval 0.089 t 0.116. Since the double degeneracy of the 120 • configuration of the the XX magnet implies a double degeneracy of the π-flux state of fermions on the triangular lattice, our approach indicates that the emergent Dirac spin-liquid state will also be doubly degenerate.
The low-energy field theory in this regime appears to be quite interesting. To understand its nature, one should integrate out fermionic degrees of freedom. In the interaction Hamiltonian, Eq. (6), only small values of momentum q contribute to the formation of the order parameter ∆ 120 • . Momenta larger than a certain momentum cutoff, q Q , are irrelevant for the low-energy description of this ordered phase. When we approach criticality near t ∼ 0.089, the fermion gap ∆ 120 • vanishes, and critical fermions with large momenta yield the Maxwell term. Indeed, the fermions do not fill any topological bands (e.g. we do not have a Chern insulator coupled to the gauge field) and integration over them will not result in generation of a Chern-Simons term [S4]. We rather have topologically trivial Dirac fermions coupled to the U (1) "probe" field. Quantum fluctuations of fermions define an effective dynamics of the gauge field, which in the leading, one loop approximation yields a 2+1 dimensional Maxwell theory. Thus at 0.116 t 0.089, one self-consistently obtains N = 6 copies of Dirac fermions interacting with the induced U (1) gauge field.