Spiral order by disorder and lattice nematic order in a frustrated Heisenberg antiferromagnet on the honeycomb lattice

Motivated by recent experiments on ${\text{Bi}}_3{\text{Mn}}_4{\text{O}}_{12}\left({\text{NO}}_3\right)$, we study a frustrated $J_1\text{−}{J}_2$ Heisenberg model on the two-dimensional (2D) honeycomb lattice. The classical $J_1\text{−}{J}_2$ Heisenberg model on the 2D honeycomb lattice exhibits Néel order for $J_2<J_1/6$. For $J_2>J_1/6$, it has a family of degenerate incommensurate spin spiral ground states where the spiral wave vector can point in any direction. Spin wave fluctuations at leading order lift this accidental degeneracy in favor of specific wave vectors, leading to spiral order by disorder. For spin $S=1/2$, quantum fluctuations are, however, likely to be strong enough to melt the spiral order parameter over a wide range of $J_2/J_1$. Over a part of this range, we argue that the resulting state is a valence bond solid (VBS) with staggered dimer order—this VBS is a lattice nematic which breaks lattice rotational symmetry. Our arguments are supported by comparing the spin wave energy with the energy of the VBS obtained using a bond operator formalism. Turning to the effect of thermal fluctuations on the spiral ordered state, any nonzero temperature destroys the magnetic order, but the discrete rotational symmetry of the lattice remains broken resulting in a thermal analog of the nematic VBS. We present arguments, supported by classical Monte Carlo simulations, that this nematic transforms into the high temperature paramagnet via a thermal phase transition which is in the universality class of the classical three-state Potts (clock) model in 2D. We discuss the relevance of our results for honeycomb magnets, such as ${\text{Bi}}_3{M}_4{\text{O}}_{12}\left({\text{NO}}_3\right)$ (with $M=\text{Mn},\text{V},\text{Cr}$), and bilayer triangular lattice magnets.

Figure 1

(Color online) Left panel: Real space basis vectors for the honeycomb lattice. Right panel: Momentum space picture depicting the manifold of classically degenerate spiral wave vectors for $J_2/J_1=0.3$ (red, thin solid), $J_2/J_1=0.5$ (purple, dash-dotted), and $J_2/J_1=0.7$ (green, dashed). Also indicated by magenta (solid) dots are the six distinct spiral wavevectors lying on this manifold which are favored by quantum fluctuations. Black (thick solid) hexagon indicates the first Brillouin zone of the lattice.

Figure 2

Plot of the spin wave correction to the energy per site $\Delta E=\left(E_{\text{qu}}-E_{\text{cl}}\right)/N$ (in units of $J_1$) as a function of $J_2/J_1$.

Figure 3

Sketch of the valence bond solid state with staggered dimer order which breaks the honeycomb lattice rotational symmetry but preserves spin rotational and lattice translational symmetries. $\widehat{a},\widehat{b}$ denote basis (unit) vectors of the triangular lattice formed by the dimer bonds. $d_{1,2}$ and ${\Delta }_{1,2}$ indicate Hartree-Fock-Bogoliubov mean field parameters in the bond operator mean field theory (see text for details). Dashed line indicates the set of spins which might be viewed as forming one dimensional dimerized chains which are coupled in the transverse direction.

Figure 4

(Color online) Ground state energy (in units of $J_1$) as a function of $J_2/J_1$. The red (solid) line is the energy of the spiral state including leading order spin wave corrections, the green (dashed) line is the nematic VBS energy up to quadratic order in triplon operators, and the blue (dash-dotted) line indicates nematic VBS energy up to quartic order in triplon operators.

Figure 5

Binder cumulant of the order parameter [from Eq. (41)] plotted as a function of temperature for various system sizes. The crossing point of the curves at $T/J_1{S}^2\approx 0.1515\left(5\right)$ indicates a continuous nematic to paramagnet phase transition at $J_2/J_1=0.8$. Inset shows the nematic transition temperature, $T_c$, as a function of $J_2/J_1$. Lines are guides to the eye.

Figure 6

Nematic susceptibility [from Eq. (42)], in units of $1/J_1$, as a function of temperature for various system sizes at $J_2/J_1=0.8$. Lines are guides to the eye. Inset shows the specific heat peak at the transition for $L=72$.