Quantum and thermal phase transitions in a bosonic atom-molecule mixture in a two-dimensional optical lattice

We study the ground state and the thermal phase diagram of a two-species Bose-Hubbard model, with $\text{U}\left(1\right)\times {\mathbb{Z}}_2$ symmetry, describing atoms and molecules on a two-dimensional optical lattice interacting via a Feshbach resonance. Using quantum Monte Carlo simulations and mean-field theory, we show that the conversion between the two species, coherently coupling the atomic and molecular states, has a crucial impact on the Mott-superfluid transition and stabilizes an insulating phase with a gap controlled by the conversion term—the Feshbach insulator—instead of a standard Mott-insulating phase. Depending on the detuning between atoms and molecules, this model exhibits three phases: the Feshbach insulator, a molecular condensate coexisting with noncondensed atoms, and a mixed atomic-molecular condensate. Employing finite-size scaling analysis, we observe three-dimensional (3D) $XY$ (3D Ising) transition when $\text{U}\left(1\right)$ (${\mathbb{Z}}_2$) symmetry is broken, whereas the transition is first order when both $\text{U}\left(1\right)$ and ${\mathbb{Z}}_2$ symmetries are spontaneously broken. The finite-temperature phase diagram is also discussed. The thermal disappearance of the molecular superfluid leads to a Berezinskii-Kosterlitz-Thouless transition with unusual universal jump in the superfluid density. The loss of the quasi-long-range coherence of the mixed atomic and molecular superfluid is more subtle since only atoms exhibit conventional Berezinskii-Kosterlitz-Thouless criticality. We also observe a signal compatible with a classical first-order transition between the mixed superfluid and the normal Bose liquid at low temperature.

Figure 1

Mean-field insulating-BEC transition for fixed density $\rho =2$ with $g/U=0.8$ close to the resonance (a) $\delta /U=-1$, (b) on the molecular side, $\delta /U=-10$, and (c) on the atomic side, $\delta /U=10$. The FI phase is stabilized by the conversion $g$ whereas the ${\mathrm{MI}}_{\mathrm{am}}$ phase is stabilized by the interactions $U$. The jump in the condensate fractions $C_a^{\text{MF}}$ and $C_m^{\text{MF}}$ at the FI-${\mathrm{BEC}}_{\mathrm{am}}$ transition indicates a first-order transition, associated with the metastable region $t/U\in ]0.061;0.065[$ where the ground-state energy exhibits global and local minima.

Figure 2

Mean-field ground-state phase diagram, taken from Ref. [21], with $t/U=0.06$ and $g/U=0.8$ [false colors indicate the compressibility $\kappa$ Eq. (10)]. The following phases appear in the phase diagrams: Feshbach insulator (FI) with $\rho =2$, molecular (${\mathrm{BEC}}_{\mathrm{m}}$) and atomic-molecular (${\mathrm{BEC}}_{\mathrm{am}}$) condensate. Second-order transitions are denoted by solid black lines, red dashed lines indicate first-order transitions, and red dots denote tricritical points.

Figure 3

Vertical cut of the mean-field phase diagram in Fig. 2 for $\delta /U=-1.0$. The ${\mathrm{BEC}}_{\mathrm{am}}$-FI transition is first order, whereas the other transitions are continuous.

Figure 4

QMC simulations of the insulating-BEC transition with $L=10$ for fixed density $\rho =2$ and $g/U=0.6, \beta t=20$ close to the resonance (a) $\delta /U=-1$, (b) on the molecular side, $\delta /U=-10$, and (c) on the atomic side, $\delta /U=10$. These results are in good qualitative agreement with the mean-field predictions of Fig. 1. (a) Close to the resonance, the conversion stabilizes the FI phase. The discontinuities in the atomic and molecular condensates, $C_a$ and $C_m$ [Eq. (11)], and in the superfluid density ${\rho }_s$ [Eq. (12)], indicate a first-order transition.

Figure 5

Finite size analysis of (a) the superfluid density ${\rho }_s$ and of the condensates (b) $C_a$, and (c) $C_m$ at the FI-${\mathrm{BEC}}_{\mathrm{am}}$ transition using QMC simulations with $\delta /U=-1, g/U=0.6, \beta t=2L$ and $\rho =2$ for linear sizes $L=8,10,12,14$. The jump at the transition increases with $L$.

Figure 6

QMC grand-canonical simulations for $L=8$ and the same other parameters as in Fig. 5. The density $\rho$ is discontinuous at the FI-${\mathrm{BEC}}_{\mathrm{am}}$ transition.

Figure 7

QMC canonical simulations at fixed total density $\rho =2$, with $g/U=0.6, \beta t=2L$, and $t/U=0.07$. (a) Atomic ${\rho }_a$ and molecular ${\rho }_m$ densities as functions of detuning: both densities jump at the first-order FI-${\mathrm{BEC}}_{\mathrm{am}}$ transition (see zoom in inset for $L=14$). A jump is also observed in the (b) superfluid density ${\rho }_s$, as well as in the (c) atomic $C_a$ and (d) molecular $C_m$ condensates when the size of the system increases.

Figure 8

QMC canonical simulations of the atomic and molecular condensates as functions of the detuning $\delta /U$ for $\rho =1, g/U=0.6, t/U=0.07$, and $\beta t=2L$ and two linear sizes $L=10$ and $L=12$. The system evolves from a ${\mathrm{BEC}}_{\mathrm{m}}$ to a ${\mathrm{BEC}}_{\mathrm{am}}$ phase when increasing the detuning $\delta /U$.

Figure 9

Scaling plots of the atomic and molecular condensates for the quantum phase transition from ${\mathrm{BEC}}_{\mathrm{m}}$ to ${\mathrm{BEC}}_{\mathrm{am}}$ with (a) $\rho =0.5$, (b) $\rho =1$, and (c) from ${\mathrm{BEC}}_{\mathrm{m}}$ to FI with $\rho =2$, as obtained via QMC simulations with $\beta t=2L$. Critical exponents of the (a, b) 3D Ising and (c) 3D $XY$ universality classes [42], cited in the boxes, are used for the scaling.

Figure 10

QMC thermal phase diagram with $\rho =2, g/U=0.6$, and $t/U=0.07$: the BEC phases existing strictly at $T=0$ become superfluid (SF) and then normal Bose liquid (NBL) when heating the system, keeping the detuning constant, whereas the FI phase crossed over the NBL phase. Black circles and blue squares are obtained using finite-size scaling analysis; see respectively Figs. 12 and 13. The green point indicates a quantum first-order transition; the dashed line indicates a possible first-order transition.

Figure 11

Superfluid density as a function of the detuning at the $\rho =2$ disordered-${\mathrm{SF}}_{\mathrm{am}}$ transition for finite temperatures with $L=10,12,14, g/U=0.6$, and $t/U=0.07$. The discontinuity in ${\rho }_s$, observed for (a) $T/U=0.035$, decreases with the temperature. No jump is observed for (b) $T/U=0.07$.

Figure 12

(a) Scaling of the molecular condensate $C_m$ and (b) superfluid density as functions of temperature for different sizes and $\rho =2, g/U=0.6, t/U=0.07$, and $\delta /U=-10$. The critical temperature extracted from scaling analysis with the BKT exponent in (a) is consistent with an anomalous $8T_{\text{BKT}}/\pi$ stiffness jump instead of $2T_{\text{BKT}}/\pi$.

Figure 13

Scaling of the (a) atomic and (b) molecular condensate as functions of temperature for different sizes with $\rho =2, g/U=0.6, t/U=0.07$, and $\delta /U=10$.