Testing Statistical Laws in Complex Systems

The availability of large datasets requires an improved view on statistical laws in complex systems, such as Zipf’s law of word frequencies, the Gutenberg-Richter law of earthquake magnitudes, or scale-free degree distribution in networks. In this Letter, we discuss how the statistical analysis of these laws are affected by correlations present in the observations, the typical scenario for data from complex systems. We first show how standard maximum-likelihood recipes lead to false rejections of statistical laws in the presence of correlations. We then propose a conservative method (based on shuffling and undersampling the data) to test statistical laws and find that accounting for correlations leads to smaller rejection rates and larger confidence intervals on estimated parameters.

Figure 1

Statistical laws and strong correlations occur simultaneously in complex systems. Main Panels: Distribution $p(x)$ [or its cumulative $F(x)$] of observable $x$ for the data (blue dots) and maximum-likelihood fit of different statistical laws (orange, see Supplemental Material [30], Sec. I). Insets: autocorrelation $C(\tau )$ with time lag $\tau$ of the observable $x$ for the original data (blue) and randomized data (orange, average and 1 or 99 percentiles over 1000 realizations); ${\tau }^{*}$ indicates the value at which the original and randomized $C(\tau )$ are statistically indistinguishable [31]. (a) Sequence of magnitudes $x$ of earthquakes in Southern California from 1981–2010 [32] ($N=59555$ with commonly used threshold $x\ge 2$ [33]). (b) Sequence of interevent times $x$ (measured in words) of consecutive occurrences of the word “the” in the book Moby Dick obtained from Project Gutenberg [34] ($N=14042$ with threshold $x\ge 3$). (c) Sequence of words (tokens) in the order they appear in the book Ulysses by James Joyce obtained from Project Gutenberg [34]; $x$ the rank of the word (type) in terms of frequency in the whole book ($N=264971$ word tokens, obtained removing punctuation and nonalphabetic characters). (d) Sequence of degrees of nodes from a network; $x$ is the rank of the degree of the node; the sequence $\{x_i\}$ used to compute $C(\tau )$ was obtained applying an edge-sampling method to the complete network (see Supplemental Material [30], Sec. II); the network corresponds to the connections between autonomous systems of the Internet [35], $V=34761$ vertices (nodes) and $E=107720$ unique edges (in our case $N$ is the number of half edges and thus $N=2E$).

Figure 2

Correlations impact the fitting of power law distributions using Maximum Likelihood methods. Two synthetic datasets following a power law with exponent $\alpha =1.5$ for $x=1,\dots ,1000$ were generated: one using independent sampling (in orange or light gray) and one with correlations (in blue or dark gray); see the Supplemental Material [30], Sec. II, for different choices of the maximum cutoff leading to similar results. (a) Distribution $p(x)$ for a single realization with $N={10}^5$. Inset: Autocorrelation function $C(\tau )$. (b) Average and 95% confidence interval of the KS distance over 100 different realizations of the synthetic data. Inset: Rejection rate, i.e., fraction of realizations for which the power law is rejected on a 0.05-significance level according to method of Ref. [17] (dotted line) for datasets of varying length $N$. (c) Average and 95% confidence interval of the estimated power law exponent $\widehat{\alpha }$ over 100 different realizations of the synthetic data.

Figure 3

Decorrelating data leads to different conclusions in hypothesis testing in artificial (a) and (b) and empirical (c)–(f) data. The distribution of $p$ values from fitting correlated (of size $N$, in blue) and subsampled shuffled data (of size $n\le N$, in orange). While the correlated data leads to a peaked distribution of small $p$ values (i.e., rejection), the decorrelated data obtained from our approach leads to the expected flat distribution of $p$ values. The effective sample size $N^{*}$ (black vertical line) was obtained from ${\tau }^{*}$ reported in Fig. 1 as $N^{*}=N/{\tau }^{*}$. While all cases are rejected when fitting the full dataset, in three out of four cases we cannot reject the null hypothesis for decorrelated data (median $p$ value $\ge 0.05$ at $n=N^{*}$).