Dynamical Computation of the Density of States and Bayes Factors Using Nonequilibrium Importance Sampling

Nonequilibrium sampling is potentially much more versatile than its equilibrium counterpart, but it comes with challenges because the invariant distribution is not typically known when the dynamics breaks detailed balance. Here, we derive a generic importance sampling technique that leverages the statistical power of configurations transported by nonequilibrium trajectories and can be used to compute averages with respect to arbitrary target distributions. As a dissipative reweighting scheme, the method can be viewed in relation to the annealed importance sampling (AIS) method and the related Jarzynski equality. Unlike AIS, our approach gives an unbiased estimator, with a provably lower variance than directly estimating the average of an observable. We also establish a direct relation between a dynamical quantity, the dissipation, and the volume of phase space, from which we can compute quantities such as the density of states and Bayes factors. We illustrate the properties of estimators relying on this sampling technique in the context of density of state calculations, showing that it scales favorable with dimensionality—in particular, we show that it can be used to compute the phase diagram of the mean-field Ising model from a single nonequilibrium trajectory. We also demonstrate the robustness and efficiency of the approach with an application to a Bayesian model comparison problem of the type encountered in astrophysics and machine learning.

Figure 1

Mean-field Ising model with potential (15): volume ratio obtained from (13) using a single ascent descent trajectory pair. (Inset) The free energy in $\beta$ and $m=d^{-1}\sum _{i=1}^d\cos (q_i)$ that can also be estimated from this single trajectory (right half) and analytically using large deviation theory (LDT) (left half).

Figure 2

Mixture of Gaussians inference problem with $d=10$ and 50 wells in the mixture. The volume of states below $E=-\log \mathcal{L}$ is shown as red circles. In the small gamma limit, the time to reach energy $E$, $\mathrm{\Delta }\tau ={\tau }^E-{\tau }^{-}$, should be independent of the initial condition. (Inset) The dissipative dynamics converges to many different local minima, corresponding to the individually discernible lines in the inset. A total of $N=100$ trajectories are plotted, but there are fewer than 100 visible lines, meaning that some trajectories not only end in the same basin, but also have decay in energy at the same rate, as the small $\gamma$ limit predicts. This demonstrates that the low variance regime can be achieved even with modest values of $\gamma$.