Exploring cluster Monte Carlo updates with Boltzmann machines

Boltzmann machines are physics informed generative models with broad applications in machine learning. They model the probability distribution of an input data set with latent variables and generate new samples accordingly. Applying the Boltzmann machines back to physics, they are ideal recommender systems to accelerate the Monte Carlo simulation of physical systems due to their flexibility and effectiveness. More intriguingly, we show that the generative sampling of the Boltzmann machines can even give different cluster Monte Carlo algorithms. The latent representation of the Boltzmann machines can be designed to mediate complex interactions and identify clusters of the physical system. We demonstrate these findings with concrete examples of the classical Ising model with and without four-spin plaquette interactions. In the future, automatic searches in the algorithm space parametrized by Boltzmann machines may discover more innovative Monte Carlo updates.

Figure 1

A schematic plot of the Boltzmann machine and its typical use in machine learning. The Boltzmann machine consists of stochastic variables (red and blue dots) and hidden units (white and gray squares) connected into a network. Different colors denote active or inactive states of various units. The joint probability distribution of these variables follows a Boltzmann distribution. By adjusting the structure and parameters of the BM, it models the target probability distribution of input data as the marginal probability distribution of visible variables. The sampling of the Boltzmann machine generates new samples according to the learned distribution. Here, we show that a BM with an appropriately designed architecture can suggest efficient cluster Monte Carlo algorithms in its generative sampling.

Figure 2

(a) The Boltzmann machine (3) reproduces cluster Monte Carlo algorithms of the Ising model (2). Solid dots residing on the vertices are the visible units representing the Ising spins. Red and blue colors denote Ising spin up and down. The squares residing on the links are the binary hidden units, where the white and gray colors indicate the inactive ($h_{\ell }=0$) or active ($h_{\ell }=1$) status of the hidden unit. The effective interaction between the visible units can either be $W$ (thick links) or 0 (thin links). (b) The sampling of the BM. Given the visible units, we sample the hidden units according to Eq. (4). The inactive hidden units (white squares) divide the visible units into disconnected components which can be flipped collectively at random.

Figure 3

The Boltzmann machine (8) suggests a new cluster update for the plaquette Ising model (6). Red/blue dots on the vertices denote the visible Ising spins, and white/gray squares in the plaquette center denote the hidden units. The double arrows point to the two parallel links ${\ell }_{\wp },{\overline{\ell }}_{\wp }$ composing the plaquette $\wp$. The hidden units are sampled directly according to Eq. (9) where the breakup of the plaquette into parallel links is chosen at random. Once the hidden units are given, Eq. (8) reduces to an inhomogeneous Ising model where the visible spins interact with modified coupling strengths, indicated by the thicknesses of the links.

Figure 4

Results for the Ising model with four-spin plaquette interactions (6) on square lattices with linear length $L$. (a) Binder ratio obtained using the cluster update suggested by BM (8) at $K/J=0.2$. The dashed line indicates the universal value for the two-dimensional Ising universality class. (b) The cluster update improves the energy autocorrelation time by orders of magnitude at the critical point compared to the local update.

Figure 5

A plaquette of the fully frustrated Ising model. The solid and dashed lines indicate ferromagnetic and antiferromagnetic coupling, respectively. The red and blue dots denote up and down Ising spins. (a) A plaquette configuration with three satisfied bonds and one unsatisfied bond, and ${\mathcal{F}}_{\wp }\left(\mathbf{s}\right)=2$. (b) A plaquette configuration with three unsatisfied bonds and one satisfied bond, and ${\mathcal{F}}_{\wp }\left(\mathbf{s}\right)=-2$. The gray/white squares in the plaquette center denote the hidden units in the active/inactive status.