Predicted and verified deviations from Zipf’s law in ecology of competing products

Zipf’s power-law distribution is a generic empirical statistical regularity found in many complex systems. However, rather than universality with a single power-law exponent (equal to 1 for Zipf’s law), there are many reported deviations that remain unexplained. A recently developed theory finds that the interplay between (i) one of the most universal ingredients, namely stochastic proportional growth, and (ii) birth and death processes, leads to a generic power-law distribution with an exponent that depends on the characteristics of each ingredient. Here, we report the first complete empirical test of the theory and its application, based on the empirical analysis of the dynamics of market shares in the product market. We estimate directly the average growth rate of market shares and its standard deviation, the birth rates and the “death” (hazard) rate of products. We find that temporal variations and product differences of the observed power-law exponents can be fully captured by the theory with no adjustable parameters. Our results can be generalized to many systems for which the statistical properties revealed by power-law exponents are directly linked to the underlying generating mechanism.

Figure 1

Complementary cumulative distribution functions (CDF) of the market share of digital cameras measured for four snapshots on (a) April 1, 2005, (b) September 22, 2006, (c) February 2, 2007, and (d) November 14, 2007. Maximum likelihood estimates of a power-law distribution are shown by the blue (continuous, upper) lines and the maximum likelihood estimates of a truncated (at the left) log-normal distribution are shown by the red (dashed, bottom) lines. In all cases, the empirical distributions go beyond the exploration of a truncated log-normal.

Figure 2

Test of Gibrat’s law of proportional growth for market share $S$ of products until January 31, 2008. The left panel [i.e., (a)] shows the test for the mean of the the increments (i.e., $\langle \Delta S\rangle$) versus its current size. The right panel [i.e., (b)] depicts the test for the standard deviation of the increments (i.e., $\Delta S$) versus its current size. In both panels the blue (continuous) line show the OLS fit to the data points. $R^2$ values are 0.913 for the left panel [i.e., (a)] and 0.984 for the right panel [i.e., (b)].

Figure 3

Distribution of initial market product shares, taking the market share value of the first Sunday occurring two weeks after the birth of a given product as a proxy. The distribution of the initial market share distribution is shown by the black circles. Blue (continuous) line shows the maximum likelihood estimate of a power-law distribution and the red (dashed) line shows the maximum likelihood of a log-normal distribution.

Figure 4

Birth flows measured at the time scale of one week. The black circles denote a three-month moving average and the red crosses denote a six-month moving average. The blue (continuous) line shows the fitted curve $\lambda \left(t\right)=5.95*[1-0.49*\sin (0.23*t\left)\right]$ and the black (dashed) curve corresponds to $\lambda \left(t\right)=5.34*\exp (0.001*t)$. Since $2*\pi /0.23=27.3\left(\text{weeks}\right)=191.13\left(\text{days}\right)$, we can see that the periodicity is approximately six month. Notice also the sharp drop of birth flows found during January 16, 2006 to August 8, 2006.

Figure 5

Scatter plots describing (a) the relationship between product lifetime and overall market share (as defined in the text) and (b) the relationship between product lifetime and the market share of products at the time of death for all the digital cameras that died during the period January 2005 to December 2007. Note that, for products that attained more than 0.5% of the market share, size independence of lifetime seems to hold.

Figure 6

The left panel [i.e., (a)] depicts the complementary cumulative distribution function (CDF) of lifetimes of digital cameras using the definition mentioned in the text. Green circles denote the distribution of lifetimes of products which died during 2005, blue squares correspond to 2006, red triangles to 2007, and black crosses mixing all of these years together. The right panel [i.e., (b)] depicts the time evolution of the market share and of the average posted price movement for a particular digital camera. The (continuous) red line depicts the market share (left vertical axis) and the black (dotted) line shows the average posted price over all retail stores (right vertical axis).

Figure 7

Scatter plot showing the relationship between drift ($r$) and volatility ($\sigma$) for the period November 2004 to December 2007. Black circles denote digital cameras, red triangles represent TV’s, blue squares code for keyboards, and green cross for mouse products. The drift and volatility for digital cameras, mouse products and TV’s were calculated by opening a 6 month time window and conditioning on when market share is above 0.5%. For keyboards, this level was slightly increased to 0.75% because, while the mode number of products sold each day during 2007 is around 3500–5000 for other products, it was only 1300 for keyboards, making 0.5% (6.5 products sold a day) pertaining to the noisy part of the distribution.

Figure 8

The left panel [i.e., (a)] depicts the complementary cumulative distribution functions of lifetimes of the mouse product. Blue circles denote the distribution of lifetimes of products that died during October 2004 to December 2005. The red crosses show the distribution of lifetimes of products that died during January 2006 to December 2007. The straight lines show the maximum likelihood estimates, assuming an exponential distribution. Only products which attained an overall market share of 0.5% or higher during their lifetimes were used when estimating the distribution of lifetimes to focus on the top-selling products. The right panel [i.e., (b)] depicts the time evolution of the volatility ($\sigma$) of market shares of digital cameras. We can see an extraordinary drop in volatility during the time window January 16, 2006 to August 8, 2006.

Figure 9

Comparison between the maximum likelihood estimate $\mu \left(\text{MLE}\right)\left(t\right)$ and our theoretical prediction $\mu \left(\text{TH}\right)\left(t\right)$ for four time series, namely (a) digital camera, (b) TV, (c) keyboard, and (d) mouse. For each panel, the dashed line shows the estimated power-law exponent obtained from the empirical distribution. The lower threshold for the validity of the power-law tail of the distribution was estimated using the method described in Ref. [31]. The red continuous line shows our theoretical exponent obtained for a running window of six-month duration. The blue dotted line shows our theoretical exponent for a running time window of one-year duration.