Light Microscopy at Maximal Precision

Microscopy is the workhorse of the physical and life sciences, producing crisp images of everything from atoms to cells well beyond the capabilities of the human eye. However, the analysis of these images is frequently little more accurate than manual marking. Here, we revolutionize the analysis of microscopy images, extracting all the useful information theoretically contained in a complex microscope image. Using a generic, methodological approach, we extract the information by fitting experimental images with a detailed optical model of the microscope, a method we call parameter extraction from reconstructing images (PERI). As a proof of principle, we demonstrate this approach with a confocal image of colloidal spheres, improving measurements of particle positions and radii by 10–100 times over current methods and attaining the maximum possible accuracy. With this unprecedented accuracy, we measure nanometer-scale colloidal interactions in dense suspensions solely with light microscopy, a previously impossible feat. Our approach is generic and applicable to imaging methods from brightfield to electron microscopy, where we expect accuracies of 1 nm and 0.1 pm, respectively.

Popular Summary

Microscopy is the workhorse of the physical and life sciences. Centuries of research in optics have provided windows into the world well beyond the limits of the human eye, telling tales from worm locomotion to protein binding. Yet the analysis of these images frequently performs little better than by human inspection. Current automated techniques rely on approaches taken from computer vision that were designed to mimic human visual perception. While rapid, these approaches contain the same biases and systematic errors present in human analyses. Worse, since the algorithms are heuristics discovered through trial and error, there is no systematic way to improve their accuracy.

Here, we present a scientific, methodological approach for analyzing microscopy images. Our technique relies on fitting experimental images to a detailed optical model of the microscope, extracting all the useful information present in the image. As a demonstration of this approach, we improve the accuracy of particle positions and sizes extracted from confocal microscopy images of dense colloidal suspensions by a factor of 10 to 100 over current methods. Finally, we use our method to measure nanometer-scale interactions in dense colloidal suspensions.

Our improvement in accuracy arises without any modification to the microscope itself, allowing researchers to improve the quality of their data by using cheap computing rather than expensive microscope modifications.

Figure 1

PERI overview. A demonstration of model information recovered from real confocal microscope images of $⟨a⟩=1.343(8)\mu \mathrm{m}$ colloidal spheres at a volume fraction of $\varphi =0.130(5)$. (a) The generative model consists of a platonic image of dye distributed around perfect spheres and a coverslip (top panels), illuminated with a spatially varying intensity (middle panels), and convolved with a physical point-spread function (bottom panels). The left panels show the model components, and the right panels show the combined image. (b) These components combine to form a realistic generative model (bottom right), which, aside from noise, is visually indistinguishable from the data (top left). (c) From the fit parameters of the model, we extract information such as particle positions and radii (orange highlights) to within a few percent of a pixel–corresponding to an accuracy of 1 and 3–4 nm, respectively. The inset zooms into the center of the particle (white square), highlighting the precise localization and calculated uncertainty of the particle’s position.

Figure 2

Fitting the generative model to experimental data. (a) A representative image (lower left), its best-fit model (center), and the difference between the two (upper right), shown as cross sections in the $xy$, $xz$, and $yz$ planes. The residuals show nearly perfect Gaussian white noise at the expected signal-to-noise ratio, demonstrating the quality of the generative model. (b) The Fourier power spectrum of the same residuals, displayed as three orthogonal slices in the $q_x{q}_y$, $q_x{q}_z$, and $q_y{q}_z$ planes. In addition to scanning noise, visible as the stripes along $q_x=0$ and $q_z=0$ as well as the isolated poles, excess power is visible at scales larger than the particles themselves but smaller than the features given by the illumination field (ILM). These residuals are associated with the incomplete description of the point-spread function. (c) PERI measures the particle radius within an uncertainty of 3–4 nm, as estimated from changes in featured particle radii with time (red histogram) measured by tracking approximately 1200 particles over 800 time points. Improving the description of the PSF would allow for radii to be featured at the CRB (green histogram), with a precision of 1 nm. (d) The experimental mean-squared displacement along the $x$ direction $⟨\mathrm{\Delta }{x}^2⟩$ (black dots; error bars are smaller than symbol size) from approximately 1200 particles measured over 800 time points provides an estimate of PERI’s average positional errors. Extrapolating the fitted mean-square displacement (red curve; the shaded band denotes the fit uncertainty) to $t=0$ gives a positional error indistinguishable from 0 nm. The mean-square displacement along $y$ and $z$ provide the same constraints.

Figure 3

Extracting interparticle potentials. (a) We use PERI to analyze a large ensemble of three-dimensional images of a dilute suspension of approximately 1200 Brownian particles; a small section of these images is shown in the upper left. From these data, we extract an experimental $P_s(\delta )$. (b) We use molecular dynamics to create a simulated $P_s(\delta )$, and we iteratively update the interaction potential $V(\delta )$ to find the $P_s(\delta )$ that best fits the experimental data. (c) The extracted $P_s(\delta )$ from PERI (gray dashed line) and from the fitted potential (solid cyan line) agree excellently. In contrast, the $P_s(\delta )$ from a strictly hard-sphere potential (red line) does not fit the data. The difference between these potentials depends on resolving particle separations at the nanometer level. Previous centroid-based methods [41] (purple line) produce a $P_s(\delta )$ with nonsensical features, such as significant overlaps, that cannot be fit by a reasonable interaction potential. We discuss the overlaps in PERI’s extracted $P_s(\delta )$ in SM [18], Sec. VII. (d) From the best-fit simulation, we extract the interparticle potential $V(\delta )$. The shaded bands show the uncertainty in the potential, with the teal band describing uncertainty in the fit and the gray band the uncertainty due to systematic errors (see SM [18] for further discussion).