Solving the inverse problem of noise-driven dynamic networks

Nowadays, massive amounts of data are available for analysis in natural and social systems and the tasks to depict system structures from the data, i.e., the inverse problems, become one of the central issues in wide interdisciplinary fields. In this paper, we study the inverse problem of dynamic complex networks driven by white noise. A simple and universal inference formula of double correlation matrices and noise-decorrelation (DCMND) method is derived analytically, and numerical simulations confirm that the DCMND method can accurately depict both network structures and noise correlations by using available output data only. This inference performance has never been regarded possible by theoretical derivation, numerical computation, and experimental design.

Figure 1

Schematic figures on forward and inverse problems of complex networks of Eq. (3). (a) Forward problem of Eq. (3): from network structure $\widehat{\mathbf{A}}$ and noise statistics $\widehat{\mathbf{Q}}$ to calculate output variable data $\mathbf{y}\left(t\right)$. (b) Typical inverse problem: from available output variable data $\mathbf{y}\left(t\right)$ and noise statistics $\widehat{\mathbf{Q}}$ to retrieve network Jacobian structure $\widehat{\mathbf{A}}$, by using matrix equation $\widehat{\mathbf{A}}\widehat{\mathbf{C}}+\widehat{\mathbf{C}}{\widehat{\mathbf{A}}}^T=-\widehat{\mathbf{Q}}$ [39, 40, 41] with $\widehat{\mathbf{A}},\widehat{\mathbf{Q}}$, and $\widehat{\mathbf{C}}$ given in Eqs. (3), (4a), and (7). (c) Another nontypical inverse problem: from available variable data $\mathbf{y}\left(t\right)$ and Jacobian structure $\widehat{\mathbf{A}}$ to depict noise statistics $\widehat{\mathbf{Q}}$. (d) A desirable inverse computational target not regarded possible so far: from available variable data $\mathbf{y}\left(t\right)$ only to reveal both network Jacobian structure $\widehat{\mathbf{A}}$ and noise statistics $\widehat{\mathbf{Q}}$. And this is the task of this paper.

Figure 2

Solving the inverse problem by double correlation matrices and noise-decorrelation (DCMND) method of algorithms (9) and (15). $N=100$. 500 active links and 500 repressive ones are randomly chosen, and $A_{ij}\left(0\right)=0$ for all other off-diagonal matrix elements. Time step for simulations of Eq. (3) is $dt=5\times {10}^{-4}$ and time interval for velocity computation is $\Delta t_q={10}^{-3}$. $L=5\times {10}^5$ samples are yielded for inferences. (a), (b) Case 1: Jacobian matrix $\widehat{\mathbf{A}}$ is given as follows: active interactions $A{\left(a\right)}_{ij}=1$; repressive interactions $A{\left(r\right)}_{ij}=-1$; the diagonal terms are set to $A_{ii}=-3$; and noise correlation matrix is given as $Q_{ij}={\sigma }_i{\delta }_{ij},{\sigma }_i=0.01$. (c), (d) Case 2: the same as (a), (b) with heterogeneous $\widehat{\mathbf{A}}$ and $\widehat{\mathbf{Q}}$, i.e., $A_{ij}\left(a\right)\in (1.0,2.0)$ and $A_{ij}\left(r\right)\in (-2.0,-1.0)$ with uniform distributions, $A_{ii}=-5$, and ${\sigma }_i\in (0.005,0.015)$ with uniform distribution. (a), (c) The trajectories in a $2\text{D}$ phase plane for the two sets of networks and parameters. (b) The distribution of off-diagonal elements of $\widehat{\mathbf{A}}=\widehat{\mathbf{B}}{\widehat{\mathbf{C}}}^{-1}$ [blue triangles for $A_{ij}\left(a\right)$, red squares for $A_{ij}\left(r\right)$, and green circles for $A_{ij}\left(0\right)$] calculated by Eq. (9) from data of (a). The computational elements of matrix $\widehat{\mathbf{B}}{\widehat{\mathbf{C}}}^{-1}$ coincide very well with the actual $\widehat{\mathbf{A}}$ with certain fluctuated errors, and all active, repressive, and null interaction natures can be accurately predicted. (d) Matrix elements computed by $\widehat{\mathbf{B}}{\widehat{\mathbf{C}}}^{-1}$ are plotted against true Jacobian matrix elements $A_{ij}$ for case 2 [all colors have the same meaning as (b)]. All dots locate closely near the diagonal line, indicating correct inferences of both interaction natures and coupling strengths. (e) Element values of $-2{\widehat{\mathbf{B}}}_s$ plotted versus element values of actual $\widehat{\mathbf{Q}}$ for both the noises of (a), (c). The agreements between $-2{\widehat{\mathbf{B}}}_s$ and $\widehat{\mathbf{Q}}$ convincingly show that the DCMND method can depict not only Jacobian matrix $\widehat{\mathbf{A}}$ but also noise correlation matrix $\widehat{\mathbf{Q}}$. The task addressed in Fig. 1 is fulfilled indeed. (f), (g) Exactly the same as (c), (d), respectively, with noise intensities increasing 100 times [${\sigma }_i\in (0.5,1.5)$ with uniform distribution]. Matrix $\widehat{\mathbf{A}}$ can be retrieved equally well. (h) Standard error (SE) of Eq. (16) plotted against the number of samples $L$. SEs decay as $\frac{1}{\sqrt{L}}$.

Figure 3

Inverse computation of the DCMND method on a cell cycle model [27] driven by white noise [Eqs. (17)]. In Eqs. (17) parameters are given as [$r=10,k_{synth}=0.16,k_{dest}=0.01,k_a=0.1,k_d=0.001,k_{wee1}=0.05,k_{wee1basal}=k_{wee1}/r,k_{cdc25}=0.1,k_{cdc25basal}=k_{cdc25}/r,cdk{1}_{tot}=230,cdc{25}_{tot}=15,wee{1}_{tot}=15,ap{c}_{tot}=50,plx{1}_{tot}=50,nwee1=4,ncdc25=4,napc=4,nplx1=4,ec{50}_{plx1}=40,ec{50}_{wee1}=40,ec{50}_{cdc25}=40,ec{50}_{apc}=40,k_{cdc25on}=1.75,k_{cdc25off}=0.2,k_{apcon}=1,k_{apcoff}=0.15,k_{plx1on}=1,k_{plx1off}=0.15,k_{wee1on}=0.2,k_{wee1off}=1.75,k_{cak}=0.8,k_{pp2c}=0.008$] and initial condition [$\mathbf{x}\left(0\right)=(0,0,0,0,0,0,wee{1}_{tot}=15,0,0)$]. $Q_{ij}={\sigma }_i{\delta }_{ij},{\sigma }_i=0.0001$. (a) The cell cycle regulatory network. Thick (thin) lines represent strong (weak) interactions. $\to$ represents active interactions and $⊸$ repressive ones. (b) All the nine variable sequences produced by Eqs. (17) available for the inverse computation. The motions of nine proteins are strongly heterogeneous, some of which have large variation amplitudes while some others show very weak oscillations. Simulation time step $dt={10}^{-3}$ and time interval for velocity computation $\Delta t_q={10}^{-3}$. (c) A $2\text{D}$ projection of the trajectory of Eqs. (17). (d) ${\left[B{C}^{-1}\right]}_{ij}$ plotted against actual $A_{ij}$ at six different phase points [shown in (c)] by applying the DCMND method of Eq. (9) to the measurable output data of (b), (c). $L=5\times {10}^5$ samples are yielded for inferences for each phase-space region $P_i$. $\left(A_{ij}\right)\mathrm{s}$ are the actual Jacobian matrix elements at the corresponding points. All dots are located around the diagonal line, confirming the validity of the DCMND method. (e) Some matrix elements computed by algorithm (9) (hollow dots) and the corresponding actual Jacobian values (solid dots) for the cell cycle system at six different phase-space points. All the computed values coincide well with actual ones with certain small fluctuated errors. (f), (g) The same as (c), (d), respectively, with noise intensities increased 25 times, and the inference results in (g) are very similar to those in (d). (h) The same as Fig. 2 with the cell cycle system considered. All samples are taken around the six space points indicated in (c), respectively. The SEs of the DCMND method decrease monotonously as $\frac{1}{\sqrt{L}}$ for all the six phase-space points.

Figure 4

Inverse computation for the chaotic Rössler systems Eq. (18). Simulation time step $dt=5\times {10}^{-4}$, velocity computation interval $\Delta t_q=5\times {10}^{-3}$, and average samples $L=5\times {10}^5$. (a), (b) Networks are randomly constructed which are asymmetric [links $A_{ij}\left(a\right)=1$ are randomly chosen with probability $p=0.10$ and $A_{ij}\left(0\right)=0$ for all other off-diagonal matrix elements]. The noise statistics in Eq. (18) is specified as $Q_{ij}={\sigma }_i{\delta }_{ij},{\sigma }_i=1.0$. Without noise the dynamic networks process synchronous chaos. The network structures can be successfully retrieved for different network sizes [(a) $N=100$ and (b) $N=200$].

Figure 5

Inverse computation for asymmetric scale-free networks with approximate power-law distributions for both in and out degree. $N=200,\langle k\rangle =13$, and all the parameters for computation are the same as Fig. 2 except $A_{ii}=-5$ and scale-free network structures applied here. The DCMND method can successfully depict the Jacobian matrix $\widehat{\mathbf{A}}$ with scale-free links, and the figure is very similar to Fig. 2 with randomly distributed interactions.

Figure 6

Comparison of the DCMND method with three commonly used inference methods introduced in Eqs. (A1), (A2), and (A3). The same data set of case 2 of Fig. 2 are used for all the four methods: DCMND method (a), Pearson correlation (b), mutual information (c), and regression (d) [all colors have the same meanings as Fig. 2]. It is clearly shown that the DCMND method is a unique method to depict network structures, from output data only, for both interaction natures and coupling strengths.

Figure 7

Same as Fig. 6 with the cell cycle model Eqs. (17) considered. The data set of $P_1$ in Fig. 3 are used for all the four methods: DCMND method (a), Pearson correlation (b), mutual information (c), and regression (d). In (a)–(d) the DCMND method is a unique method to depict network structures, from output data only, for both interaction natures and coupling strengths. The dots of all the three commonly used methods deviate from the diagonal line considerably, indicating incorrect inferences of interaction strengths. In particular, some wrong conclusions of interaction natures can be observed in (b), (c), and (d).