Log-Log Convexity of Type-Token Growth in Zipf’s Systems

It is traditionally assumed that Zipf’s law implies the power-law growth of the number of different elements with the total number of elements in a system—the so-called Heaps’ law. We show that a careful definition of Zipf’s law leads to the violation of Heaps’ law in random systems, with growth curves that have a convex shape in log-log scale. These curves fulfill universal data collapse that only depends on the value of Zipf’s exponent. We observe that real books behave very much in the same way as random systems, despite the presence of burstiness in word occurrence. We advance an explanation for this unexpected correspondence.

Figure 1

Type-token growth curve $v(\ell )$ for three random systems with number of counts drawn from a discrete power-law distribution $N_L(n)\propto n^{-\gamma }$, and $\gamma =1.8$ (green diamonds), 2.0 (red circles), and 2.2 (blue triangles). The black lines correspond to our theoretical predictions, Eq. (9) for $\gamma \ge 2$ and Eq. (10) for $\gamma <2$ (plotted with the help of the GNU Scientific Library). No average over the reshuffling procedure is performed. Curves are consecutively shifted by a factor of $\sqrt{10}$ in the $x$ axis. (Inset) The ratio $v(\ell )/{\ell }^{\alpha }$ is displayed, with $\alpha =\min \{1,\gamma -1\}$, showing that an approximation of the form $v(\ell )\propto {\ell }^{\alpha }$ is too crude.

Figure 2

The rescaled vocabulary-growth curve of 28 books from the PG database with exponents $\gamma =\{1.8,2.0,2.2\}\pm 0.01$ fitted for $n\ge 1$ or $n\ge 2$. The values of $L$ and $V$ range from 27873 to 146845 and from 5639 to 30912, respectively. As is apparent, all books with the same exponent collapse into a single curve, which Eqs. (9) and (10) accurately capture. For the case of Eq. (10), we have used a value of $n_{\max }/L=0.05$. The dotted straight lines (shifted for clarity) indicate the behavior predicted by Heaps’ law.

Figure 3

Distribution of the rescaled interoccurrence distances ${\widehat{\tau }}_k$; see Eq. (13). The scale parameter $⟨{\tau }_k⟩$ was computed from the data of each book [types with $n=1$ or with $N(n)=1$ were not included in the analysis]. The original books (red) display clear deviations from an exponential distribution for $k>1$, but not for $k=1$. Shuffled versions of the books (green) do not show, as expected, any clustering effect, and hence their rescaled interoccurrence distances are roughly exponentially distributed. (Top panel) The book Moby Dick, by Herman Melville, as an illustrative example. (Bottom panel) The 100 longest books in the PG database.