Feature learning and network structure from noisy node activity data

In the studies of network structures, much attention has been devoted to developing approaches to reconstruct networks and predict missing links when edge-related information is given. However, such approaches are not applicable when we are only given noisy node activity data with missing values. This work presents an unsupervised learning framework to learn node vectors and construct networks from such node activity data. First, we design a scheme to generate random node sequences from node context sets, which are generated from node activity data. Then, a three-layer neural network is adopted training the node sequences to obtain node vectors, which allow us to construct networks and capture nodes with synergistic roles. Furthermore, we present an entropy-based approach to select the most meaningful neighbors for each node in the resulting network. Finally, the effectiveness of the method is validated through both synthetic and real data.

Figure 1

The node vectors trained from the first case study are visualized via PCA. The five communities have an equal number of nodes. Panels (a–d) are, respectively, the training results of $Data.1$ to $Data.4$.

Figure 2

The node vectors trained from the second case study are visualized via the PCA. The five communities have 1000, 1500, 500, 750, and 1250 nodes. Panels (a–d) are, respectively, the training results of $Data.1$ to $Data.4$.

Figure 3

Visualization of node vectors trained from the first example with 20% missing values. Panels (a–d) are, respectively, the training results of $Data.1$ to $Data.4$.

Figure 4

The visualization of node vectors trained from the Pearson correlation network. Panels (a–d) are, respectively, $Data.1$ to $Data.4$.

Figure 5

The properties of the two real datasets. Panels (a) (first dataset) and (b) (second dataset) are the distributions of the CV of the expression values. Panels (c) (first dataset) and (d) (second dataset) are the distributions of means of the expression values.

Figure 6

Visualization of the gene vectors via t-SNE, the genes are colored by the expression values. Panels (a) (first dataset) and (b) are the visualizations of node vectors trained from raw data. Panels (c) (first dataset) and (d) (second dataset) are the visualizations of node vectors trained from the incomplete datasets with 20% values randomly removed from each condition.

Figure 7

The visualization of node vectors trained from the PCC network.

Figure 8

Experimental results when different thresholds are applied. (a) shows the percentages of isolated nodes w.r.t. the thresholds. (b) shows the edge densities of networks when different thresholds are used.

Figure 9

Edge density analysis of the REM. (a) Synthetic data. (b) Real data.

Figure 10

The degree distributions of networks generated from the GTE and REM. Panels (a) and (b) are the degree distributions of the networks constructed from synthetic data. Panels (c) and (d) are the degree distributions of the networks constructed from real data. Panels (a) and (c) show the results of densely connected networks, while panels (b) and (d) are the results of sparsely connected networks.

Figure 11

The comparison of node degrees between densely and sparsely connected networks.

Figure 12

The three-layer neural network model. Each input node is associated with an $N$-dimensional one-hot vector [47], which is mapped to the node vector $f\left(v_i\right)$ (the hidden layer) of dimension $d$ by matrix $W_1$. The hidden layer is mapped to the output vector by matrix $W_2$. The elements of $W_1$ and $W_2$ are initialized with random values, which are expected to be optimized by backpropagation [48].