Network dynamics for optimal compressive-sensing input-signal recovery

By using compressive sensing (CS) theory, a broad class of static signals can be reconstructed through a sequence of very few measurements in the framework of a linear system. For networks with nonlinear and time-evolving dynamics, is it similarly possible to recover an unknown input signal from only a small number of network output measurements? We address this question for pulse-coupled networks and investigate the network dynamics necessary for successful input signal recovery. Determining the specific network characteristics that correspond to a minimal input reconstruction error, we are able to achieve high-quality signal reconstructions with few measurements of network output. Using various measures to characterize dynamical properties of network output, we determine that networks with highly variable and aperiodic output can successfully encode network input information with high fidelity and achieve the most accurate CS input reconstructions. For time-varying inputs, we also find that high-quality reconstructions are achievable by measuring network output over a relatively short time window. Even when network inputs change with time, the same optimal choice of network characteristics and corresponding dynamics apply as in the case of static inputs.

Figure 1

Reconstruction of input images. (a), (c), (e): Original input images; (b), (d), (f): reconstructions using the network dynamics and a 10:1 input signal component to output neuron ratio, with relative errors $0.1885,0.1756,$ and $0.2206$, respectively. (g) Reconstruction of the image in (e) using static CS, yielding a relative error of $0.1497$. Images (a) and (c) are of size $100\times 100$ pixels, and image (e) is of size $200\times 200$ pixels. We choose $s\left(A\right)=0.95$, and the optimal sparsity of $s\left(B\right)=0.999$ for reconstructions of images (a) and (c) with $n=10000$ input components. For reconstructions of image (e) with $n=40000$ input components, we choose the appropriate optimal sparsity of $s\left(B\right)=0.99975$. In the reconstructions of image (e), $m=4000$ neurons are utilized, maintaining a 10:1 ratio of input components to neurons (samples for the static CS reconstruction).

Figure 2

Reconstruction error and network dynamics dependence on input drive strength. (a) Reconstruction error as a function of network input strength, $f$. (b)–(d) Average voltage plots for $f=0.3,1,$ and 5, respectively.

Figure 3

Network connectivity and interaction strength. (a) Reconstruction error as a function of neuron interaction strength $S$. (b) Reconstruction error as a function of the sparsity of network connectivity matrix $A$. (c), (d) Raster plots for $S=1000$ and $S=1$, respectively.

Figure 4

Reconstruction error and network dynamics dependence on input sampling. (a) Reconstruction error as a function of the sparsity of the signal sampling matrix, $s\left(B\right)$. Inset zooms into the highly sparse regime near $s\left(B\right)=1$. (b)–(d) Average voltage plots for $s\left(B\right)=0.1,0.999,$ and $0.99999$, respectively.

Figure 5

Interspike interval statistics dependence on input sampling. (a) Variance of ${\tau }_{\text{ISI}}$ across all firing events in the network as a function of the sparsity, $s\left(B\right)$, of input signal sampling matrix $B$. (b)–(d) ${\tau }_{\text{ISI}}$ histograms for $s\left(B\right)=0.1,0.999,$ and $0.99999$, respectively.

Figure 6

Comparison of average voltage periodicity. (a), (b) Log-log plots for the power spectral density (PSD) of the average voltage dynamics for networks with $s\left(B\right)=0.1$ and $s\left(B\right)=0.999$, respectively. Each PSD was computed over the course of $20000$ ms with 200 time windows. (c), (d) Correlation functions corresponding to the PSDs in considered in (a) and (b), respectively.

Figure 7

Periodicity and randomness of network dynamics. (a) Correlation time ($T$) as a function of the sparsity of $B$. (b) Entropy ($H$) of the interspike intervals across all network firing events as a function of the sparsity of $B$. Insets zoom into the highly sparse regime near $s\left(B\right)=1$.

Figure 8

Robustness of CS network input reconstruction. (a) Reconstruction error as a function of the ratio of input components to the number of neurons in the network, $n/m$, for a fixed input with $n$ components. (b) Reconstruction error dependence on the variance of injected Gaussian noise perturbing the input components.

Figure 9

Reconstruction of smoothly changing input. (a) Sequence of input image frames, each injected into the network for $t_f=200$ ms each. (b) Network CS reconstruction of the images depicted in (a). The average reconstruction error is $0.1678$.

Figure 10

Reconstruction and dynamics for nonsmoothly changing input. (a) Top: Sequence of injected image frames. Bottom: Reconstruction of images for an injection time of $t_f=200$ ms each. The average reconstruction error is $0.1512$. (b) Average voltage plot for the network over the time course of the injection of the images in (a). (c) Firing rate agreement between the prediction of the linear mapping given by Eq. (4) (blue), prediction of the nonlinear mapping given by Eq. (3) (red), and the simulated network (green) averaged over the entire time-course of the simulation. In each case, we note that there is close agreement between the firing rates of individual neurons, with many overlapping points. (d) Reconstruction error as a function of the duration of each image injection following the initial image.