Non-Line-of-Sight Three-Dimensional Imaging with a Single-Pixel Camera

Real-time high-resolution three-dimensional (3D) reconstruction of scenes hidden from the direct field of view is a challenging field of research, with applications in real-life situations related, e.g., to surveillance, self-driving cars, and rescue missions. Most current techniques recover the 3D structure of a non-line-of-sight (NLOS) static scene by detecting the return signal from the hidden object on a scattering observation area. Here, we demonstrate the full color retrieval of the 3D shape of a hidden scene by coupling back-projection imaging algorithms with the high-resolution time-of-flight information provided by a single-pixel camera. By using a high-efficiency single-photon avalanche-diode (SPAD) detector, this technique provides the advantage of imaging with no mechanical scanning parts, with acquisition times down to the subsecond range.

Figure 1

The schematics of the experimental setup for NLOS imaging. (a) A pulsed laser scatters on a scattering surface, producing a spherical wave in all the surrounding area. The DMD is imaging a $50\times 50{\mathrm{cm}}^2$ area of the scattering surface and the time-resolved single-pixel camera collects the signal back-scattered from the target in the hidden scene by temporal histograms. We collect the signal scattered back by the hidden target by sequentially collecting light from each of the $20\times 20$ pixels either by raster-scan masks or by Hadamard masks. (b) The noncooperative single-object scene used for ultrafast NLOS imaging.

Figure 2

A cooperative two-object scenario for a NLOS single-pixel camera based on a PMT detector. (a) The return signal produced by the two hidden objects and retrieved by raster-scan imaging onto the DMD and collection of the signal on the single-pixel detector in the time frame $t=7.1\mathrm{ns}$. (c),(d) 3D shape retrieval of the hidden scene in (c) the $x$-$y$ plane and (d) the $x$-$z$ plane after applying a back-projection imaging algorithm. The dotted line represents the actual position of the targets. To facilitate the visualization, the reflectivity is normalized to the local maxima value. The objects to be recovered are two round targets of 2.54 cm and 7.62 cm diameter at a varying depth and tilted. The voxel dimension is $0.2\times 0.2\times 1{\mathrm{cm}}^3$ for an area to investigate of $20\times 20\times 100{\mathrm{cm}}^3$. The threshold used for the retrieval is 0.89.

Figure 3

A noncooperative objects RGB-colored scenario for a NLOS single-pixel camera based on a PMT detector. The red color corresponds to the recovered target by using the red spectral filter and so on. (a) The return signal produced by the hidden object by raster-scan acquisition. (b) The RGB-colored target. (c),(d) RGB retrieval of the 3D shape of the hidden objects in (c) the $x$-$y$ plane and (d) the $x$-$z$ plane after applying a back-projection imaging algorithm. The red color corresponds to the recovered target by using the red spectral filter and so on. The dotted black line represents the actual position of the target. We discretize the space in voxels of $1.4\times 1.4\times 1{\mathrm{cm}}^3$ for an overall area of $140\times 140\times 100{\mathrm{cm}}^3$.

Figure 4

The ultrafast NLOS single-pixel camera results. (a) The return signal produced by the hidden object by Hadamard acquisition at a time frame of 5.3 ns. (b) The return signal obtained by the preprocessing filter. (c),(d) Retrieval of the 3D shape of the hidden target in (c) the $x$-$y$ plane and (d) the $x$-$z$ plane after applying a back-projection imaging algorithm. The dotted black line represents the actual position of the target. The object to be recovered is a rectangular target of $24\times 10{\mathrm{cm}}^2$. The voxel dimension is $1.4\times 1.4\times 1{\mathrm{cm}}^3$ for an overall of $140\times 140\times 100{\mathrm{cm}}^3$. The threshold used for the retrieval is 0.80.