State Concentration Exponent as a Measure of Quickness in Kauffman-type Networks

We study the dynamics of randomly connected networks composed of binary Boolean elements and those composed of binary majority vote elements. We elucidate their differences in both sparsely and densely connected cases. The quickness of large network dynamics is usually quantified by the length of transient paths, an analytically intractable measure. For discrete-time dynamics of networks of binary elements, we address this dilemma with an alternative unified framework by using a concept termed state concentration, defined as the exponent of the average number of t-step ancestors in state transition graphs. The state transition graph is defined by nodes corresponding to network states and directed links corresponding to transitions. Using this exponent, we interrogate the dynamics of random Boolean and majority vote networks. We find that extremely sparse Boolean networks and majority vote networks with arbitrary density achieve quickness, owing in part to long-tailed in-degree distributions. As a corollary, only relatively dense majority vote networks can achieve both quickness and robustness.


I. INTRODUCTION
Networks of binary elements are useful tools for investigating a plethora of dynamical behavior and information processing in biological and social systems. For example, various models of associative memory are used to study neural information processing [1][2][3]. Random Boolean networks, also known as the Kauffman nets, show rich dynamics and are used to model gene regulation [4][5][6]. Random majority vote networks are often used to understand mechanisms for ordering in neural information processing [3,[7][8][9], gene regulation [6], and collective opinion formation in social systems [10].
Properties desirable for dynamics of networks of such binary units include robustness and quickness. A system is defined to be robust when flipping a small number of units' states does not eventually alter behavior of the entire network. For the random Boolean networks, the robustness has been quantified in the context of damage spreading in cellular automata [11][12][13][14].
Dynamics are usually called quick if an orbit starting from an arbitrary state reaches the corresponding attractor within a small number of steps on average, i.e., with a short transient length of the dynamics. It is known that the random Boolean networks show long-tailed distributions of the transient length, implying that the transient is long on average [15]. However, analytical evaluation of the transient length is difficult. Therefore, we theoretically study the quickness of dynamics by introducing a new concept of state concentration, extending the previous statistical dynamical framework [7][8][9]. In particular, the exponent of concentration, which we introduce later, is an analytically tractable quantity to measure the quickness of dynamics in the random Boolean and majority vote networks.
Using this exponent, we investigate the compatibility of the robustness and quickness in these two types of networks.
For this purpose, we distinguish densely connected Boolean network (DBN), sparsely connected Boolean network (SBN), densely connected majority vote network (DMN), and sparsely connected majority vote network (SMN). We show that strong state concentration, accompanied by a power law type of the indegree distribution with exponential cutoff, occurs in the majority vote networks (DMN and SMN) but not for the Boolean networks except for extremely sparse cases. Then, we argue that the DMN is the only type among the four types of networks that realizes both robustness and quickness.

II. MODEL
Let us consider a network of n binary units. We define discrete-time dynamics of the network by where x i (t) ∈ {1, −1} is the binary state of the ith unit at time t. For a random Boolean network, each f i is randomly and independently chosen one of the 2 2 n Boolean functions on the n units. For a majority vote network, where sgn indicates the sign function. We consider an ensemble of randomly generated majority vote networks where w ij are independently and identically distributed Gaussian random variables. In general, a constant or random threshold could be included in the above dynamical expression, which we omit here for simplicity. If the value of f i (x 1 , . . . , x n ) depends only on randomly chosen K units for each i, the model is called the K-sparse network [4,5]. The DBN and DMN correspond to K ∝ n, and the SBN and SMN correspond to K ≪ n. We study typical dynamical behavior of the random DBN, SBN, DMN, and SMN.

III. DISTANCE LAW IN STATE TRANSITION
The network state at time t is given in vector form as Let X = {x} be the set of the N ≡ 2 n states. Given a network, the state transition is a mapping from X to itself. We write it briefly as The dynamics of the distance between two state trajectories has been studied to characterize dynamics in these networks. We define the normalized Hamming distance between two states x and y in X by It should be noted that the distance is restricted to the range 0 ≤ D(x, y) ≤ 1. If d ′ = ϕ(d) holds true for a function ϕ(d) for any x and y almost always as n → ∞, where d = D(x, y) and d ′ = D (f x, f y), we call it the distance law. For the DBN, ϕ(d) = 0 (d = 0) and ϕ(d) = 1/2 (d = 0). For the SBN [16], For the DMN [7][8][9], For the SMN [17], where K j is the binomial coefficient and The dynamics of the distance under annealed approximation are given by

IV. EXPONENT OF STATE CONCENTRATION
The transient length before the orbit enters the attractor is analytically intractable. To quantify the quickness of the dynamics by using an alternative order parameter called the exponent of state concentration, we use the so-called state transition graph [4,5,19] defined as follows. A map f , either Boolean or majority vote, induces a graph on N nodes. Each state x ∈ X defines a node and has exactly one outgoing link directed to node f x.
Suppose that each of the N = 2 n nodes (i.e., states) has a token at t = 0. For each t(≥ 0), an application of f moves all the tokens at each node x to node f x. Repeated applications of mapping f elicit concentration of tokens at specific nodes. We denote by f −t x the set of nodes whose tokens move to x after t steps and by |f −t x| the number of tokens at node x after t steps. Tokens are initially equally distributed, i.e., |f 0 x| = 1 for each x, and the total number of tokens is conserved, i.e., for t ≥ 0.
The indegree of node x in the state transition graph is equal to |f −1 x|. The nodes with where ∅ is the empty set, do not have parent nodes. The set of such nodes is called the garden of Eden [4]. and denoted by E 1 . The nodes x ∈ E 1 only appear as initial state. Only the nodes x ∈ X − E 1 receive tokens at t = 1. In general, we define , the set of nodes that do not own tokens at time step t. There where T is the longest transient period and E * is the set of the transient states. The set of the attractors is given by A = X − E * (Fig. 1). To quantify the state concentration, we consider the frequency with which a randomly selected token at x(0) and t = 0 meets other tokens after applying f . We write the rela- (1) and that x(0) is selected with equal probability (i.e., 1/N). In general, we denote by S t the expected number of t-fold parent nodes of a node x(t) conditioned by a state transition path ending at x(t) through which a token has traveled, i.e., Obviously S 0 = 1 and sequence {S t } is monotonically nondecreasing in t. If holds true for large n, the tokens are exponentially concentrated on nodes having at least a t-fold parent node. We refer to We define i.e., the indegree distribution of node x(1) conditioned by x(0) → x(1). The symmetry guarantees that r k is independent of x(0) and x(1). Let us compute We denote by y(0) a node such that D (x(0), y(0)) = d for a given x(0). The number of nodes with distance d away from x(0) is given by where is the entropy. The probability that D (f x(0), f y(0)) = d ′ (see Fig. 2 for a schematic illustration of this situation) is given by In particular, By using the saddle-point approximation, we obtain where and To evaluate c t in general, we consider a t-step state transition path X t = {x(0) → x(1) → · · · → x(t) = x * } ending at x * and calculate the conditional probability that another path Y t = {y(0) → · · · → y(t)} ends at the same x * . S t is the expectation of the number of such t-step paths. Let us denote the distance D(x(t ′ ), y(t ′ )) by d t ′ , where 0 ≤ t ′ ≤ t and d t = 0.
Then, under the Markov assumption, the probability of path Y t conditioned by path X t is represented in terms of the distances of the two sequences, i.e., d t ′ , 0 ≤ t ′ ≤ t, by x(0) For example, for t = 2, we obtain where On the basis of the expression of ϕ for the SBN and SMN shown before, the dependence of c 1 , c 2 , c 3 , and c 4 on K is plotted in Fig. 3. For the SBN, c t (1 ≤ t ≤ 4) converges to 0 quickly as K increases. For the DBN, which is the case for K = n, we trivially obtain c t = 0 at least for small t because f is equivalent to the random mapping. Figure 3 indicates that the state concentration occurs only for very small K in the random Boolean network. In contrast, the state concentration occurs even for large K in the majority vote network. In particular, for the DMN with K → n, we obtain c 1 ≈ 0.157 [9]. Figure 3 also indicates that the state concentration quickly proceeds as t increases, except in the DBN. We verified Eq. (13) by comparing S t obtained from direct numerical simulations (i.e., Eq. (11)) and c t generally given by Eq. (25). The results shown in Fig. 4 indicate that the theory (lines) seems to agree with numerical results at least for large N; although the largest N value shown in the figure is only n = 25. Therefore, the Markov assumption (Eq. (23)), also implicitly assumed for t = 2, 3, and 4 in Fig. 3, roughly holds true up to t ≈ 4 for large n.
Theoretically, most sequences Y t that meet X t after t steps of state transition own the sequence of distance given by d * In particular, a majority of the initial states Y 0 is initially separated from X 0 by d * 0 (t). Figure 4 suggests that this is the case at least up to t ≈ 4 for large n. The sequence of distance d * t is shown for 1 ≤ t ≤ 4 in

V. INDEGREE DISTRIBUTION OF THE STATE TRANSITION GRAPH
We calculate the incoming degree distribution where k is the indegree of a state and N k=0 p k = 1. Because each node in the state transition graph has exactly one outgoing link, we have where · indicates the expectation. Because r k = kp k [7] (also see [20] for an example), we obtain Therefore, c 1 > 0 indicates that k 2 diverges in the limit of N = 2 n → ∞, reminiscent of the scale-free property of the state transition graph [20][21][22].
For the DBN, the state transition graph is the directed random graph in which p k obeys the Poisson distribution (i.e., p k = 1/ek!) with mean and variance 1 [4,5]. Therefore, k 2 = 2, proving that c 1 = 0 for the DBN (i.e., no exponential state concentration). This is consistent with Fig. 4(a) (circles). Figure 3 suggests that c 1 ≈ 0 when K is approximately larger than 10. Therefore, the degree distribution of the state transition graph is also narrowly distributed for the SBN with K ≥ 10. We verified that the numerically obtained indegree distribution for the random Boolean network with n = 30 and K = 20 approximately obeys the Poisson distribution ( Fig. 6(a)).
In contrast, a positive value of c 1 found for the SBN with small K, DMN, and SMN ( Fig. 3) indicates that k 2 (= S 1 ) diverges exponentially in n. This is actually the case, as shown in Fig. 4(b, c, d). For scale-free networks with p k ≈ k −γ , the extremal criterion would lead to γ ≈ (c 1 + 3 log 2)/(c 1 + log 2) [21,22]. However, the degree distribution numerically obtained for the DMN, shown in Fig. 6(b), deviates from a power law. The degree distribution numerically obtained for the SBN is also different from a power law [19].
To guides to the eyes, a fitting curve on the basis of a power law with an exponential cutoff is shown by the line in Fig. 6(b). In fact, the power law is not the only distribution that yields the divergence of k 2 . In the present case, the position of the exponential cutoff may mildly diverge as N becomes large.

VI. DISCUSSION AND CONCLUSIONS
In summary, we provided a unified framework for analyzing the state concentration. We found that the state concentration occurs in the SBN with small K, DMN, and SMN, but not in the DBN. We also revealed the long-tailed distributions of the indegree of the state transition graph in the SBN, DMN, and SMN, but not in the DBN.
We briefly discuss the relationship between the quickness, measured by the exponent of state concentration in this study, and the robustness of the dynamics. The robustness of the dynamics is often measured in terms of damage spreading. It is a long-term property concerning the stability of d = 0 for mapping ϕ. As we mentioned, d = 0 is an unstable fixed point of ϕ unless K = 1 or 2. Although the SMN with K = 1 or 2 satisfies quickness and robustness, we do not discuss these cases because the dynamics in these cases are just frozen