Optimal Design of Experiments by Combining Coarse and Fine Measurements

In many contexts, it is extremely costly to perform enough high-quality experimental measurements to accurately parametrize a predictive quantitative model. However, it is often much easier to carry out large numbers of experiments that indicate whether each sample is above or below a given threshold. Can many such categorical or “coarse” measurements be combined with a much smaller number of high-resolution or “fine” measurements to yield accurate models? Here, we demonstrate an intuitive strategy, inspired by statistical physics, wherein the coarse measurements are used to identify the salient features of the data, while the fine measurements determine the relative importance of these features. A linear model is inferred from the fine measurements, augmented by a quadratic term that captures the correlation structure of the coarse data. We illustrate our strategy by considering the problems of predicting the antimalarial potency and aqueous solubility of small organic molecules from their 2D molecular structure.

Figure 1

Combining coarse and fine measurements accurately predicts antimalarial activity and solubility. The predictive accuracy of (a) ${\mathrm{pIC}}_{50}$ against malaria and (b) solubility as a function of the number of quantitative measurements given to the model with coarse measurements (blue line) and without (red line). Including random quadratic terms (orange line) is not effective; error bars were obtained over 30 random partitions of data into training (90%) and verification (10%) sets. (Inset) An out-of-sample solubility prediction with 90% of the full data set has a mean absolute error of 0.61 ($r^2=0.85$). The estimate is arrived at by analyzing ten random partitions of the data into training and verification sets.

Figure 2

Hopfield patterns can be recovered from threshold sampling. (a) The histogram of eigenvalues agrees with the Marčenko-Pastur distribution (red curve) save for three significant eigenvalues. (b) The top eigenvector is the lowest-energy Hopfield pattern; the other eigenvectors are shown in Supplemental Material [24]. Random matrix cleaning allows us to successfully (c) recover the coupling matrix $J$ and (d) predict Hopfield energies.

Figure 3

Hopfield inference with random matrix cleaning is robust to the energy threshold. Here $\rho$ is the Pearson correlation coefficient between the entries in $J$ and $J_{\mathrm{MP}}$. The results are computed by averaging over 50 realizations.

Figure 4

Few quantitative measurements enable $J$ to be inferred accurately for stratified data sets. (a) There are now four significant eigenvectors but still only three Hopfield patterns in the model. (b)–(d) The top eigenvector is uncorrelated with all Hopfield patterns, and the Hopfield patterns are demoted to the second to fourth significant eigenvectors. (e) Random matrix cleaning does not recover the coupling matrix. (f) Blue data points: The elements of the coupling matrix recovered by incorporating quantitative data and using Eq. (3). Orange data points: The elements of the coupling matrix recovered using the quantitative data only and ridge regression (with $J_{ij}$ being the coefficients).

Figure 5

For energy landscapes where the basin depth is not proportional to the basin width, the eigenvectors indicate the locations of energy minima, but the eigenvalues are awry. (a) There are three significant eigenvectors above the Marčenko-Pastur threshold. (b)–(d) The top eigenvector is correlated with the highest-energy minimum, and the last significant eigenvector is correlated with the lowest-energy minimum.