Machine Learning Link Inference of Noisy Delay-Coupled Networks with Optoelectronic Experimental Tests

We devise a machine learning technique to solve the general problem of inferring network links that have time delays using only time series data of the network nodal states. This task has applications in many fields, e.g., from applied physics, data science, and engineering to neuroscience and biology. Our approach is to first train a type of machine learning system known as reservoir computing to mimic the dynamics of the unknown network. We then use the trained parameters of the reservoir system output layer to deduce an estimate of the unknown network structure. Our technique, by its nature, is noninvasive but is motivated by the widely used invasive network inference method, whereby the responses to active perturbations applied to the network are observed and employed to infer network links (e.g., knocking down genes to infer gene regulatory networks). We test this technique on experimental and simulated data from delay-coupled optoelectronic oscillator networks, with both identical and heterogeneous delays along the links. We show that the technique often yields very good results, particularly if the system does not exhibit synchrony. We also find that the presence of dynamical noise can strikingly enhance the accuracy and ability of our technique, especially in networks that exhibit synchrony.

Popular Summary

In many systems of individual dynamic units, the units interact among themselves and, in turn, affect each other’s dynamics. Examples range from neurons connected by axons to Facebook profiles connected by friendships. In many cases, the interactions between the units can be imagined to be occurring along a network of links. Determining this interaction network is thus a key step toward understanding the behavior of these systems. Here, we formulate and test a new approach to inferring an interaction network in the common situation where the dynamics is noisy and the cause-and-effect interactions among units are delayed.

Building on recent developments in machine learning, we propose a two-step method for noninvasively inferring the network of connections, using solely the measured time-series data from the dynamics of the network-connected units. In the first step, we train an artificial neural network to mimic the observed time evolution of the system. In the second step, we track the spread of disturbances in that trained neural network and then use that information to infer the network interaction structure of the original system. We also test our technique on experimental and simulated time-series data from optoelectronic networks—an excellent test bed for complex dynamics—and show that our technique is extremely effective.

Based on this result, we anticipate that this new method for inferring an interaction network offers the promise of widespread future impact for the study of dynamics on such networks.

Figure 1

Schematics of the RC trained for predicting the time series $k$ time steps ahead. Lower: the four time series represent scalar components of $\mathcal{X}[t]$.

Figure 2

Illustration of optoelectronic oscillator and coupling scheme. Red lines indicate signal paths in optical fibers. Black lines are used to indicate electronic signals, and green indicates the DSP board.

Figure 3

Examples of experimental and simulated time series from two optoelectronic oscillator networks showing globally synchronous (upper) and completely desynchronized (lower) behavior, respectively. For the simulations, we use the noise strength $\kappa ={10}^{-6}$. In the plots, purple overlays orange, which overlays red, which overlays blue.

Figure 4

List of possible connected directed four-node networks [65] with different numbers of links ($L$).

Figure 5

Synchronization error for simulated time series of the networks in Fig. 4 realized with optoelectronic oscillator nodes with different coupling strengths for random initial conditions. The color-coded synchronization error for each of the 62 networks in Fig. 4 is shown as one of the 62 horizontal bars for each value of the coupling strength. Here, the convention we follow is that, for a fixed number of links ($L$), moving upward, the horizontal bars correspond to the networks listed in Fig. 4 left to right. The same convention is followed in Figs. 6, 7, and 10.

Figure 6

Simulation test results with varying noise strength $\kappa$. (a) Number of false positives and (b) synchronization error for simulated time series from different networks with progressively increasing noise. As described in the text, each horizontal cut of the plots represents a single trajectory of the system, starting from a random initial condition. The convention for the sequence of the networks is the same as in Fig. 5.

Figure 7

Simulation test results with varying coupling strength $\epsilon$. (a) Number of false positives and (b) synchronization error for simulated time series from different networks with progressively increasing coupling strength. As described in the text, each horizontal cut of the plots represents a single trajectory of the system, starting from a random initial condition. The convention for sequence of the networks is the same as in Fig. 5.

Figure 8

This figure shows statistics of our results for the link scores $S_{i,j}$ of all the possible networks (listed in Fig. 4) with $\epsilon =0.6$ and $\kappa ={10}^{-3}$, where, in determining the statistics, for each of the networks, we use a single random realization of the initial condition and reservoir couplings. In (a), for each individual directed node pair $(i,j)$, i.e., each candidate link, a point is plotted in the sync-error or score plane with true links plotted in blue and link absences denoted in red, with red overlaying blue. The two black horizontal lines divide the sync-error or score plane into three regions corresponding to networks that are highly desynchronized, moderately synchronized, and strongly synchronized. (b)–(d) show histograms of the node scores for each of the three levels of network synchronization demarcated in (a). Bins with scores that all correspond to true links (absence of links) are colored blue (red). Bins with scores corresponding to both true links and link absence are vertically stacked into upper and lower pieces, where the lower piece (blue) corresponds to the number of true links in the bin and the upper piece (red) corresponds to the number of missing links.

Figure 9

Typical performance of our network inference method on experimental data. Each of the vertical bars corresponds to the experimental realization of a distinct optoelectronic oscillator network, with the height indicating the corresponding number of links. Each bar is separated into three segments as shown, but in many of them only two are seen if there is a perfect link inference. The numbers inside the segments indicate the corresponding heights of the segments, rounded to the nearest integers. Ranges of coupling strength ($\epsilon$) and synchronization error for all the examples are indicated as well.

Figure 10

Tests on networks with heterogeneous link delays. (a) Average number of false positives and (b) the corresponding synchronization error for different networks listed in Fig. 4, with $\epsilon =0.6$ and $\kappa ={10}^{-6}$, and different amounts of link delay heterogeneity, presented as the ratio of link delay variation range $2\mathrm{\Delta }\tau$ and mean link delay ${\tau }_0$. The results are based on simulated data with different network configurations and different random link delays along the network links. In each case, the averaging is done over 100 different random realizations of reservoir connections, initial conditions, and assignments of interaction delays to different links. The convention for sequence of the networks is the same as in Fig. 5.

Figure 11

Time delay determination by cross-correlation of measured time series. In all cases, the coupling delay time is 1.44 ms. (a) The network to be inferred from time series measurements. (b) The cross-correlation between the time series of nodes 1 and 2. A strong peak is observed near $-1.44\mathrm{ms}$. (c) The cross-correlation between the time series of nodes 2 and 3. A strong negative peak is observed near 1.44 ms and a strong positive peak near $-1.44\mathrm{ms}$. (d) The cross-correlation between the time series of nodes 2 and 4. A strong peak is observed near 1.44 ms. (e) The cross-correlation between the time series of nodes 1 and 4. Peaks are observed at 1.44 and $-1.44\mathrm{ms}$, even though there is no link from node 1 to node 4.