Steady state and mean recurrence time for random walks on stochastic temporal networks

Random walks are basic diffusion processes on networks and have applications in, for example, searching, navigation, ranking, and community detection. Recent recognition of the importance of temporal aspects on networks spurred studies of random walks on temporal networks. Here we theoretically study two types of event-driven random walks on a stochastic temporal network model that produces arbitrary distributions of interevent-times. In the so-called active random walk, the interevent-time is reinitialized on all links upon each movement of the walker. In the so-called passive random walk, the interevent-time is only reinitialized on the link that has been used last time, and it is a type of correlated random walk. We find that the steady state is always the uniform density for the passive random walk. In contrast, for the active random walk, it increases or decreases with the node's degree depending on the distribution of interevent-times. The mean recurrence time of a node is inversely proportional to the degree for both active and passive random walks. Furthermore, the mean recurrence time does or does not depend on the distribution of interevent-times for the active and passive random walks, respectively.


I. INTRODUCTION
A broad range of diffusive processes on networks, from consensus formation [1,2] to current flows on electric circuits [3], can be modeled by random walks or, equivalently, by Markov chains. Their unbiased exploration of the underlying structure also makes them popular tools for designing algorithms for, e.g., navigation and search on networks [4][5][6], defining central nodes in a given network [7][8][9], community detection [10], and respondentdriven sampling [11,12]. In tandem with these applications, the impact of network structure on dynamics of random walks, including the hitting time, mixing time, and stationary density has been extensively studied.
However, recent studies identified limitations of the classical network paradigm, where dynamics is modeled by a dynamical process on a static underlying structure. In a broad range of empirical systems, evidence suggests instead that dynamics presents non-trivial correlations between events and long-tailed interevent time distributions [13][14][15]. These observations are incompatible with the Poissonian statistics implicitly assumed in stochastic models, therefore calling for richer models for temporal networks [16].
An important practical question concerns the impact of the temporality of a network on diffusion. In order to address this question, a first approach consists in simulating random walks on real or synthesized data of time-stamped event sequences, and to compare their dynamical properties to those of properly defined null models [17][18][19][20]. A second approach consists in studying analytically the properties of random walks on specific models of temporal networks. In most studies, however, network structure changes at regular time intervals, and transitions between networks at different times are independent [18,21,22] or Markovian [23,24].
In the present work, we follow the latter path to analytically model diffusion under the non-Poissonian nature of interevent-times. Previous studies proposed to model temporal networks as stochastic sequences of events obeying a prescribed distribution of intereventtimes attached on each link [25][26][27][28]. A random walk process, called the active random walk, was then defined as a renewal process, i.e., after a walker arrives at a node, the intereventtimes attached to all links incident to the node are reinitialized [25,26,28]. Despite its non-Markovianity owing to the fact that the rate at which an event takes place depends on the time of the previous event, this stochastic process can be described by a generalized master equation, and some of its properties, such as the stationary density [25] and the relaxation time [28], were analytically solved. In this work, we will first derive an analytical expression for the mean recurrence time for the active random walk. Then, we consider a different non-renewal process, called the passive random walk, in which interevent-times are reset only on the links traversed at each jump. The passive random walk is considered to be a more natural model for diffusion on networks than the active random walk, because in reality a diffusing entity, e.g., a virus, does not typically initiate interactions between agents (i.e., nodes) upon a jump. We will use the fact that the passive random walk shows a stronger non-Markovianity than the active random walk does because passive random walkers remember their past trajectories to some extent. Finally, we will perform numerical simulations to test our analytical predictions and compare the stationary density and mean recurrence time between the two types of walks.

II. MODEL
We consider an undirected network on N nodes. We denote the set of nodes by V = {1, 2, . . . , N} and the set of links by E. We often refer to this network as the aggregate network because it is considered as the aggregation of the temporal network, which we introduce in the following, across time. Stochastic temporal networks [25,26] add a time dimension to the aggregated networks by assigning a random interevent-time τ e to each link e ∈ E in a renewal manner. The interevent-time is the interval between two consecutive activation events of the link. We denote the probability density function (PDF) of τ e by ψ e (t). We assume that the mean of τ e is finite.
By construction, the random walker jumps from the current node to a neighboring node at the instant when a link appears between the two nodes. We analyze two versions of random walks on stochastic temporal networks [26].
The first version is the so-called active random walk. In the active random walk, when the random walker arrives at a node, it reinitializes the interevent-times on all links, which makes the process renewal. In this case, the waiting-time, i.e., the time for which a walker waits on a node before the link appears, is equivalent to the interevent-time [29,30] The second version is the so-called passive random walk, which does not assume the reinitialization at all links. When the random walker moves to a neighbor through link e, a new interevent-time is drawn only for e. The complexity of the passive random walk lies in its non-renewal nature. In other words, transition rates of the passive random walk depend on the trajectory that the walker has taken such that we have to account for the entire trajectory of the random walker to accurately evaluate its behavior. It should be noted that for exponentially distributed interevent-times the active and passive random walks are identical and reduce to the usual continuous-time random walk on the aggregate (i.e., static) network.

III. ACTIVE RANDOM WALK
The steady state of the active random walk was derived through a master equation approach in Ref. [26]. In this section, we first review these results (Secs. III A and III B).
Then, we derive the mean recurrence time for the active random walk (Sec. III C).

A. Probability flows
We denote the probability that the random walker is located at node i (1 ≤ i ≤ N) at time t by p i (t). The normalization is given by N i=1 p i (t) = 1. The rate at which the walker arrives at node j from node i at time t is denoted by q j←i (t). The transition rate for a single walker to move from i to j at time t is given by r j←i (t) := q j←i (t)/p i (t). The master equation that governs the random walk is given by If the underlying static network is connected, which we assume in the following, the random walk is mixing and the stationary density, denoted by p * i := lim t→∞ p i (t), is obtained if we set lim t→∞ dp i /dt = 0 for all nodes i.
For exponentially distributed interevent-times, the transition rate is given by where τ (ij) denotes the mean of τ (ij) , the interevent-time on link (ij). By combining Eq. (1) and r j←i (t) = r i←j (t), we conclude that the steady state is the uniform distribution [31].
For arbitrary interevent-time distributions, we cannot usually calculate r j←i (t). In addition, r j←i (t) may not be symmetric with respect to i and j so that the steady state may deviate from the uniform distribution. To calculate the steady state in this case, we define f (t; j ← i) as the rate at which the walker transits from i to j after time t has elapsed since the walker arrived at i. This event happens when, link (ij) is activated at time t and any other link (ik), where k = j, has not been activated by t. Because all τ (ik) 's, with the case k = j included, are reinitialized at the arrival of the walker at node i, we obtain We use Eq. (3) to derive the master equation for the active random walk. The rate at which the random walker reaches node j from an adjacent node i at time t satisfies where is the rate at which the walker arrives at node i at time t from an arbitrary neighbor, p j←i (0) are initially chosen weights on the links satisfying and δ(t) is Dirac's delta function. A detailed proof for Eq. (4) is found in Ref. [26]. By substituting Eq. (4) in Eq. (1), we obtain for any t > 0. We define i.e., the PDF of the time to transit from node i to somewhere. In other words, f (t; i) is the PDF of min j;(ij)∈E τ (ij) , which is the first time at which a link incident to i is activated. By integrating Eq. (7) and abbreviating which is the probability to remain at i for a time longer than t, we obtain for any t > 0. The derivation of Eq. (10) is shown in Appendix A.
B. Steady state

General case
The steady state of the active random walk is evaluated via the Laplace transform of Eqs. (4) and (10), expansion of the exponential, and application of the final value theorem [26]. Here we briefly present a slightly modified derivation of the steady state. We take the Laplace transform of Eq. (10) to obtain Here,p i (s) = According to the final value theorem, the steady state probability for node i, denoted by p * i , is given by p * i = lim s→0 sp i (s). By combining Eqs. (11) and (12), we obtain where q * i := lim t→∞ q i (t) is the rate at which the random walker arrives at node j in the steady state.
and F a :=F a (0), whereF a (s) = ∞ 0 F a (t)e −st dt is the Laplace transform of matrix F a (t). We transform Eq. (4) into the Laplace space to obtain whereq(s) = (q 1 (s), . . . ,q N (s)) ⊤ and p(0) = (p 1 (0), . . . , p N (0)) ⊤ . We multiply both sides of Eq. (15) with s and obtain By taking the limit s → 0 on both sides of Eq. (16), we find that vector q * := (q * 1 , . . . , q * N ) ⊤ is the dominant eigenvector of F a , i.e., Suppose that the random walker just arrived at a node. F a contains the probabilities to make a transition from one node to another in one step, F 2 a contains the probabilities to do so in two steps, and so on. Equation (17) implies that q * is proportional to the steady state of the discrete-time random walk on the aggregate network with transition probabilities defined by F a . It should be noted that q * is unique up to the scaling factor because the aggregate network has been assumed to be connected. To obtain p * in Eq. (13), we weight the steady state vector of the discrete-time random walk with the mean time for which the walker stays at the node.

Identical distributions
When interevent-times for different links are identically distributed according to ψ(t), where d i is the degree of node i. By combining Eqs. (17) and (18), we obtain which is consistent with the fact that the steady state of the simple random walk on an arbitrary static undirected network is proportional to the degree [3,32]. By substituting Eq. (19) in Eq. (13) and using N i=1 p * i = 1, we obtain where τ ℓ 's are i.i.d. copies of the interevent-time. It should be noted that depends solely on the degree of a node. Therefore, the steady state depends only on the node's degree.
If the interevent-time is exponentially distributed, we obtain min ℓ=1,...,d i τ ℓ ∝ 1/d i so that the steady state is the uniform distribution. This is consistent with our previous argument in Section III A, where we derived this fact directly from the master equation [see Eq. (1)]. Otherwise, min ℓ=1,...,d i τ ℓ is not necessarily proportional to 1/d i such that the steady state may not be the uniform distribution.

C. Mean recurrence time
Let T i|i be the first-passage time, i.e., the time at which a random walker starting at i returns to i for the first time. We denote the PDF of T i|i by g(t; i|i). Our goal in this section is to determine the mean recurrence time given by The hopping rate of the walker generally depends on the time already spent at a node.
Therefore, we confine ourselves to the first-passage time since the walker has just arrived at node i. We will adapt the derivation of the mean first-passage time for discrete random walks on static undirected networks [8] to the case of the active random walk.
Denote by p i|i (t) the probability that the random walker is located at node i at time t given it started at node i. We obtain The first term on the right-hand side of Eq. (23) governs the case in which the random walker has not left i until time t. The second term accounts for the walker that has left i at least once. By transforming Eq. (23) into the Laplace space, we obtain Equation (24) impliesĝ where The mean recurrence time of node i satisfies By substituting Eq. (25) in Eq. (27), we obtain Application of the final value theorem yields By using the rule of L'Hospital, we obtain Furthermore, we recall thatφ i (0) = min ℓ =i;(iℓ)∈E τ (iℓ) < ∞ [see Eq. (12)], so that the tail By substituting Eqs. (12), (13), (29), (30), and (31) in Eq. (28), we obtain In particular, if interevent-times on different links are identically distributed, the combination of Eqs. (19) and (32) yields It should be noted that for discrete random walks on undirected static networks the mean recurrence time is also inversely proportional to the node's degree [8].

IV. PASSIVE RANDOM WALK
In this section, we evaluate the steady state and mean first-passage time of the passive random walk. The difference from the active random walk is that the move of a walker does not reinitialize interevent-times except on the link used for that jump. When the passive random walker arrives at a node, links incident to the node have thus already been inactive for some random time. Therefore, in contrast to the case of the active random walk, When the arrival of a walker on a node and the activation of an link are independent processes, it is known that the waiting-time distribution ρ e (t) is given in terms of the intereventtime distribution ψ e (t) by and that the average waiting-time depends on the variance of the interevent-time, a property called the waiting-time paradox or bus paradox in queuing theory [29]. In general, the time at which the random walker jumps to an adjacent node depends on the previous trajectory of the walker. However, we assume that Eq. (34) holds true in the following analysis.

A. Probability flows
In the following, we perform an approximation in order to analytically evaluate the Eq. (34). The PDF of the time when the walker transits from node i to node j given that it arrived at node i from node k is approximated by Equation (35) suggests that the trajectory of the random walker impacts the order of link activation. In particular, when the interevent-time obeys a long-tailed distribution, a transition from node i back to node k is more likely than that from i to other nodes. It should be noted that the probability of transition does not depend on the destination node in the case of the active random walk. It should be also noted that Eq. (35) is exact for exponentially distributed interevent-times.
Similarly to Eq. (4), we obtain where the initial condition satisfies Eq. (6). Similarly to Eq. (9), we define which is the approximate probability that the walker stays at node j for time longer than t given that it arrived from node i. By substituting Eq. (36) in Eq. (1), using Eq. (37), and performing a calculation similar to Appendix A for the derivation of Eq. (10), we obtain for any t > 0. Equations (36) and (38) govern the dynamics of the passive random walk.
B. Steady state

General case
To calculate the approximate steady state distribution, we proceed similarly to the case of the active random walk. By taking the Laplace transform of Eq. (36), we obtain In terms of the vectors, Eq. (39) is written aŝ whereF p (s) is the Laplace transform of the 2|E| × 2|E| matrix (|E| is the number of links in the aggregate network) given by q p (s) is the Laplace transform of the 2|E|-dimensional column vector q p (t) := (q i←j (t)) (ij)∈E , and p p (0) is the Laplace transform of the 2|E|-dimensional column vector p p (0) := (p i←j (0)) (ij)∈E . In Eq. (41), we define f (t; j ← i|ℓ ← k) ≡ 0 for i = ℓ because such a transition is impossible.
By multiplying both sides of Eq. (40) by s and letting s → 0, we obtain where q * p := (q * i←j ) (ij)∈E = (lim t→∞ q j←i (t)) (ij)∈E and F p =F p (0). To determine the steady state p * i , we transform Eq. (38) to the Laplace space and multiply both sides by s to obtain By setting s → 0 in Eq. (43), we obtain Finally, Eq. (37) implieŝ Therefore,φ i←j (0) is the (approximate) mean time for which a walker arriving at i from node j waits before moving to a neighbor.
In summary, the steady state is approximately given by where q * i←j is the solution of Eq. (42), and the normalization is given by N i=1 p * i = 1.

Identical distributions
Denote the components of F p by (F p ) (ji),(ℓk) for (ij), (kℓ) ∈ E. When interevent-times for different links are identically distributed according to ψ(t), which we assume in this section, we obtain The first equality in Eq. (47) follows from the fact that (F p ) (ℓk),(ji) > 0, indicating the walker moved from j to i and then from ℓ to k, if and only if ℓ = i. The second equality follows from the assumption that ψ e (t) = ψ(t) for any e ∈ E. Equation (47) indicates that F p is a doubly stochastic matrix. Therefore, the solution of Eq. (42) is given by q * p ∝ 1, where 1 represents the 2|E|-dimensional column vector whose all elements are equal to unity. By using Eq. (5), we obtain This result is the same as that for the active random walk with identical interevent-time distributions [see Eq. (19)].
To evaluate the right-hand side of Eq. (46), we use whereτ k 's are i.i.d. copies of the waiting-time distributed according to a common PDF ρ(t).

By substituting
In the last equality in Eq. (51), we used integration by parts. Equation (51) implies Therefore, the mean time that the passive random walker spends at a node before transiting to a neighboring node does not depend on ψ(t) except for the dependence on τ . By combining Eqs. (5), (46), (48), and (52) and using N i=1 p * i = 1, we obtain Unlike for the active random walk [see Eq. (20)], the (approximated) steady state of the passive random walk is the uniform distribution for any ψ(t) and network structure. This is also consistent with the case when interevent-times obey the exponential distribution, for which we derived the steady state in Section III A directly from the master equation.

C. Mean recurrence time
To evaluate the mean recurrence time for the passive random walk, we denote by T i←j|i the time at which a random walker leaving node i returns to i through link (ji) for the first time. The PDF of T i←j|i is denoted by g(t; i ← j|i). We also define p i|i←j (t) as the probability that the random walker is located at node i at time t, given that it arrived at i from j at time 0. It should be noted that Bayes' rule results in The PDF of the first recurrence time satisfies Here, φ i (t) denotes the probability that the walker resides at node i for time longer than t and is given by By following the same steps as in Eqs. (25)-(32), we obtain By substituting Eqs. (52) and (53) in Eq. (59), we obtain It should be noted that the mean recurrence time is also inversely proportional to the degree for the active random walk [see Eq. (32)]. It should be also noted that T i|i is independent of ψ(t) for the passive random walk except for the factor τ , but not for the active random walk.

V. EXAMPLES
To illustrate the theoretical results derived in Sections III and IV, we analyze examples in this section. We assume that the interevent-times are identically distributed for all links according to ψ(t), which is either a power-law or Weibull distribution.

A. Power-law distributed interevent-times
Consider the case in which all interevent-times follow a power-law distribution given by where α > 2, corresponding to the assumption τ < ∞. We plot the PDFs of the powerlaw by the dotted line in Fig. 1. In fact, many real data show α ≈ 1 or 1.5 [13][14][15], which apparently contradicts our choice α > 2. Here, for simplicity we assume α > 2 to investigate the effect of long-tailed distributions on the random walk on temporal networks. For the active random walk, the PDF of the transition time is calculated from the substitution of Eq. (61) in Eq. (3) as follows: The probability to make a transition from node i to an adjacent node j [see Eq. (14)] is given by The mean time for which the random walker stays at node i before moving to a neighbor [see Eq. (12)] is given by For the passive random walk, we substitute Eq. (61) in Eq. (34) to obtain the PDF of the waiting-time as follows: By substituting Eq (65) in Eq. (35), we obtain The approximate probability to make a transition from node i to a neighbor j conditioned that the random walker reached node i through link (ik) [see Eq. (41)] is given by To illustrate the difference between the active and passive random walks, we consider the network composed of three nodes shown in Fig. 2.
For the active random walk, the mean time to stay at node i, i.e., Eq. (64), is reduced to min ℓ=1,...,d i where τ ℓ 's are i.i.d. copies of the interevent-time. By substituting Eq. (72) in Eq. (20), we obtain p * a,i ∝ where subscript "a" here and in the following corresponds to the active random walk.
Equation (73) implies that the steady state is not uniform, which is in contrast to the case of exponentially distributed interevent-times. In particular, p * a,i is large for nodes with small degrees, which is opposite to the case of the discrete-time simple random walk in undirected networks for which the steady state is proportional to the node's degree [3,32]. In contrast, for the passive random walk, the steady state is the uniform distribution [see Eq. (53)].
To test our theory, we carried out numerical simulations. We set α = 3 and used the Barabási-Albert scale-free network [33] with N = 50 nodes. The two parameters in the Barabási-Albert model were set to m 0 = m = 2. We used the same single realization of the aggregate network for both active and passive random walks. We calculated the steady state as the average time for which the walker spent on each node between t = 10 3 and t = 10 8 .
The choice t = 10 3 is to exclude the transient. We initially placed the walker at one of the two nodes initially created in the Barabási-Albert algorithm, each with probability 1/2.
The numerically obtained steady state probability of each node is shown in Fig. 3(a) for the active (circles) and passive (triangles) random walks. The nodes on the horizontal axis are shown in the ascending order of the degree. The numerical results are accurately predicted by the theory (lines). In particular, the approximations made for analyzing the passive random walk do not cause a notable discrepancy between the numerical and theoretical results.
Next, we examine the mean recurrence time. For the active random walk, we substitute Eqs. (20) and (64) in Eq. (32) to obtain For the passive random walk, by substituting τ = (α − 2) −1 in Eq. (60), we obtain where subscript "p" corresponds to the passive random walk.
In the numerical simulations of the passive random walk, we assume that the walker arrived at the starting node i from each neighbor of i with the same probability at t = 0.
To mimic the steady state of the stochastic temporal network, we assumed that the initial is drawn from ψ(t). For each starting node i, we averaged the recurrence time over 10 5 realizations of the random walk.
Numerically obtained mean recurrence times on the same scale-free network as that used in Fig. 3(a) is shown in Fig. 3(b). The theory (lines) accurately matches the numerical results (symbols). We also confirmed that the numerical results when ψ(t) is the exponential distribution with the same mean (i.e., τ = 1; shown in Fig. 1 by the solid line) completely overlap with those for the passive random walk when ψ(t) is the power-law distribution [hence not shown in Fig. 3(b)]. Figure 3(b) also indicates that, for each node, the active random walk realizes a smaller mean recurrence time than the passive random walk does.

B. Weibull distributed interevent-times
For the power-law distribution of interevent-times, the steady state probability of a node decreases with the degree for the active random walk, and the mean recurrence time at each node is larger for the passive than active random walk (Sec. V A). However, these results are not universal.
To show this, we consider the case in which the interevent-time obeys the Weibull distribution given by where m (0 < m < ∞) and λ(> 0) are parameters. The Weibull distribution with m = 2 and λ = √ π/2, which yields τ = 1, is shown by the dashed line in Fig. 1. It should be noted that the tail of the distribution is shorter than that of the exponential distribution with the same mean (solid line).
We start by illustrating the dynamics of the passive random walk on the three-node network shown in Fig. 2. By combining ρ(t) = e −(λt) m with Eq. (35), we obtain independent of the λ value. For m = 1, the interevent-times are exponentially distributed such that (F p ) (21), (12) = (F p ) (23),(32) = 1/2. For 0 < m < 1, we obtain 1/2 < (F p ) (21),(12) = (F p ) (23),(32) < 1 such that the random walker tends to alternate between two nodes, which is similar to the dynamics when ψ(t) is the power-law distribution (Sec. V A). For m > 1, we obtain 0 < (F p ) (21),(12) = (F p ) (23),(32) < 1/2 such that the random walker tends to avoid The numerical results for the Weibull distribution with m = 2, λ = √ π/2, and the same scale-free network as that used in the previous section are shown in Fig. 4. The steady state probability for the active random walk increases with the degree [circles in Fig. 4(a)]. In fact, a direct calculation of Eq. (20) for the Weibull distribution yields p * i ∝ √ d i , as shown by the lines overlapping with the circles in Fig. 4(a). This result is in contrast to the case of the power-law distribution of interevent-times, for which p * i decreases with d i [see Eq. (73)]. For the passive random walk, the steady state obeys the uniform distribution (triangles), which is consistent with the theory.
The mean recurrence time under the Weibull distribution of interevent-times is shown in Fig. 4(b). The results for the passive random walk (triangles) are indistinguishable from those for the power-law and exponential distributions, which is consistent with the theory.
We also find that, for any node, the mean recurrence time is larger for the active than passive random walk. This result is opposite to that for the power-law distribution of intereventtimes.

VI. CONCLUSIONS
We studied two models of random walks on stochastic temporal networks. Our main findings are summarized as follows. First, the steady state for the passive random walk with identically distributed interevent-times on links is uniform for any network and distribution of interevent-times. Second, for the active random walk, the steady state probability decreases and increases with the degree for the power-law and Weibull distribution of interevent-times, respectively. Third, the mean recurrence time for both types of walks is inversely proportional to the node's degree. Fourth, the mean recurrence time for the passive random walk does not depend on the distribution of interevent-times. Fifth, the active random walk produces smaller and larger mean recurrence times for each node than the passive random walk does when the interevent-time obeys the power-law and Weibull distributions, respectively.
The present result that the mean recurrence time is inversely proportional to the node's degree is consistent with that in Ref. [17]. In particular, both studies conclude that the distribution of interevent-times does not affect the mean recurrence time (squares and diamonds in Fig. 7 in Ref. [17]). We reached this conclusion by explicit derivation of the mean recurrence time. In contrast, we consider that the strength of Ref. [17] in this respect lies in numerically showing the universality of this result across different data sets. It should be noted that a discrete-time simple random walk on a different temporal network model yields different results; the mean recurrence time decreases but is not inversely proportional to the degree [21]. The passive random model induces a correlated random walk. Interesting connections of the present study may be made to seminal work on correlated random walk on lattices [34][35][36] and to recent work modeling empirical pathways on networks by second-order Markov processes [19,37,38]. Pursuing connection to anomalous diffusion on lattices [39] may be also interesting. acknowledges the support provided through CREST, JST.