Compressive sensing as a paradigm for building physics models

The widely accepted intuition that the important properties of solids are determined by a few key variables underpins many methods in physics. Though this reductionist paradigm is applicable in many physical problems, its utility can be limited because the intuition for identifying the key variables often does not exist or is difficult to develop. Machine learning algorithms (genetic programming, neural networks, Bayesian methods, etc.) attempt to eliminate the a priori need for such intuition but often do so with increased computational burden and human time. A recently developed technique in the field of signal processing, compressive sensing (CS), provides a simple, general, and efficient way of finding the key descriptive variables. CS is a powerful paradigm for model building; we show that its models are more physical and predict more accurately than current state-of-the-art approaches and can be constructed at a fraction of the computational cost and user effort.

Figure 1

(a) A sparse signal (blue line) like that of Eq. (2), uniform samples of the signal at the Nyquist frequency (red dots), and a few random samples (black circles). The signal is composed of only three nonzero frequencies. (b) Exact recovery of the frequency components of the signal using compressive sensing.

Figure 2

$\parallel J_{\mathrm{exact}}-J_{\mathrm{fit}}{\parallel }_1$(solid) and $\parallel J_{\mathrm{fit}}{\parallel }_0$(dashed) vs log${}_{10}\mu$ for the short-ranged pair model with $M=200$ (a) and $M=400$ (b). Random uniform noise of $\sim$$10\%$(blue circles), $20\%$(green squares), and $50\%$ (red “x” s) of the noiseless energies was added to the fitting structures. (c) $\parallel J_{\mathrm{exact}}-J_{\mathrm{fit}}{\parallel }_1$ vs the number of fitting structures and the noise level. Each point represents an average over $\sim$100 different subsets of $M$ structures.

Figure 3

Root-mean-square errors for the prediction set (black line with empty squares) and the leave-one out cross-validation score (LOOCV, solid blue line) as functions of the parameter $\mu$. LOOCV has been averaged over ten randomly drawn sets of 100 (400) structures, and the error bars were calculated from the variance in the predicted LOOCV scores over these sets. Predictive errors for the holdout set and the fitting errors for the training set were averaged over 500 different sets of 100 (400) structures; the corresponding error bars are smaller than the size of the symbols.

Figure 4

Results from compressive sensing and leave-one-out cross validation for the fcc-based, Ag-Pt alloy system. The solid line gives the root-mean-square (RMS) errors for predictions made on a constant holdout set for CS(box and whisker) and leave-one-out cross validation (squares). The dashed lines give the ${\ell }_1$-norm of the solution vector for both methods.

Figure 5

Comparison of the interaction coefficients found using the DO method implemented in atat software and compressive sensing. The upper pane shows a comparison of two typical fits from CS and atat. The lower pane shows the coefficients that were found to be statistically relevant from both methods. The $x$ axis is the cluster radius, which is defined as the average distance from the center of mass of all cluster vertices, given as a fraction of the lattice parameter. The cluster interactions are given in meV/cluster. (Blue dots were placed on the $x$ axis even for clusters not found to be relevant to help the reader know the ordinal number of the relevant clusters.) Physical intuition suggests that shorter-radius, fewer-vertex clusters are the most important contributors in alloy energetics. Pair interaction coefficients found by both methods are similar. As the number of vertices increases, CS finds coefficients in harmony with physical intuition, while DO finds spurious, long-ranged three- and four-body interactions. CS solutions also demonstrate a convergence to one specific solution as the size of the fitting set increases. (Note: triplets and quadruplets are shown on a scale from $-$20 to 20 meV, different from the scale used for the pairs.)

Figure 6

Predicted CSCE formation energies obtained using the ECI's shown in Fig. 5; error bars are standard deviation due to different random choices of $\le$400 structure subsets. Black solid line denotes the convex hull calculated from the average energies; only Ag, Ca${}_7$Ge-type Ag${}_7$Pt (barely, with a depth of less than 1 meV/atom), $L{1}_1$ AgPt, and Pt are predicted to be $T=0$ K ground states.

Figure 7

Predictive performance of CS for protein energetics in the zinc-finger structure (shown in the inset).