R´enyi Entropy of Zeta-Urns

We calculate analytically the R´enyi entropy for the zeta-urn model with a Gibbs measure definition of the micro-state probabilities. This allows us to obtain the singularities in the R´enyi entropy from those of the thermodynamic potential, which is directly related to the free energy density of the model. We enumerate the various possible behaviours of the R´enyi entropy and its singularities, which depend on both the value of the power law in the zeta-urn and the order of the R´enyi entropy under consideration


I. INTRODUCTION
Diversity, and how to measure it, has been a subject of fundamental interest in mathematical biology and ecology for many years [1][2][3][4][5][6][7][8][9].There have been interesting contributions from numerous authors that make use of ideas from statistical mechanics and thermodynamics, specifically those related to various notions of entropy.A prototypical problem is to quantify the diversity of an ecosystem whose organisms may be divided into N distinct species, where the σth species has a relative abundance of p σ , so From a statistical mechanical perspective p σ is the probability of having a micro-state σ in some ensemble.Given the p σ there are a multitude of entropy-like measures of diversity that one might consider (and which have already been proposed), a small selection being: Species richness : Shannon entropy [10] : Gini-Simpson Index [11] : Tsallis Entropy (of order λ) [12] : Rényi Entropy (of order λ), H λ [13] : The parameter λ is a non-negative real number.In the limit λ → 1, the Rényi entropy, H λ , [13] reproduces the Shannon entropy, H 1 = − σ p σ log p σ , [10] and in the limit λ → 0, it gives the logarithm of the species richness (i.e.logarithm of the number of micro-states): H 0 = log σ p 0 σ = log σ 1.
We will focus on the Rényi entropy here.Its exponential, the diversity or Hill number [14] of order λ, denoted by where we have defined an abundance vector p = (p 1 , p 2 , p 3 , . . ., p N ), is perhaps a more suitable choice than the entropy itself in an ecological setting since the resulting Hill numbers generally have a direct interpretation in terms of familiar quantities.For instance, D 0 (p) will be the number of distinct species and is the inverse participation ratio.Also, in the uniform case D λ (1/N, 1/N, . . ., 1/N ) = N , ∀λ, giving the number of species.In essence, the Hill numbers and their generalizations are providing an "effective number of species" for an ecosystem with some input from our prejudices on the importance of rare species determined by the parameter λ.The parameter λ can be thought of as tuning the sensitivity of the diversity measure D λ (p) to the occurrence of rare species.
Since the summands are given by p λ σ , rare species (smaller p σ ) will be weighted less strongly as λ is increased.The highest sensitivity to rare species is therefore given by λ = 0.
It is possible to further refine (complicate!) such models by introducing a measure of the similarity between Z σν between species σ and ν, with 0 ≤ Z σν ≤ 1, where Z σν = 0 is total dissimilarity and Z σν = 1 is total similarity [5].In this case the Hill numbers would be modified to where We shall consider only the case Z = I here.It has been observed [15,16] that if the p σ are given by a Gibbs measure with the partition function defined by Z(β) = σ exp(−βE σ ), then the Rényi entropy H λ is related to the logarithm of the ratio of partition functions This may also be written as a difference of free energies, where the expression is the square brackets may be regarded as a q-derivative of F defined by (with λ playing the role of q).We recover the usual relation between the Shannon entropy and the free energy in the limit of λ → 1.This relation is also the basis of using the Rényi entropy and the replica trick in conformal field theory [17] and numerical [18] calculations to evaluate the entanglement entropy of various statistical mechanical systems.

II. THE MODEL
It is tempting to use simple explicitly solvable statistical mechanical models to investigate the properties of diversity measures such as the Rényi entropy and, indeed, this has already been done in [19] for the class of models, zeta-urns, which we address here.However, our aims and also our notion of a "species"/micro-state, are rather different from those of [19] as we highlight below.
Zeta-urn models describe weighted partitions of S balls (particles) between N boxes, such that each box i contains at least one particle, s i ≥ 1, and S = s 1 + s 2 + . . .+ s N .In our case micro-states σ in this model correspond to the particle distributions in the boxes σ = (s 1 , s 2 , . . ., s N ).The number of states for S particles in N boxes is N = S−1 N −1 .The abundance vector (p 1 , ...., p N ) is thus not the same as in [19], where the authors considered the abundance vector (p 1 , ...., p S ) in the set {1, 2, ..., S} in an ensemble in which the number of boxes N was allowed to fluctuate.The geometrical picture behind this choice in [19] is of breaking a bar of length S into N segments of sizes (s 1 , s 2 , . . ., s N ) and maximizing the diversity (by some measure) of these, which was then applied to the general problem of partitioning a set of S elements into N components with a power-law distribution for the probabilities of the component sizes.In [19] it was found using a phenomenological calculation based on cluster size estimation in spin models [20] that D 0 was maximized for β = 2, which the is the value given by Zipf's law [21,22].This, and the predicted scaling with S, agreed well with numerical simulations.
Here we would like to use (7,8) to investigate the singular behaviour of the Rényi entropies in a zeta-urn model.To this end, the energy of the system in the state σ is taken to be logarithmic in the number of particles in each box The corresponding partition function [23,24] Z S,N (β) = with for s = 1, 2, 3, . ... The parameter β in the power law for the weights can thus be considered as the inverse temperature: β = 1/T .Despite its simplicity, this model occurs in many problems of statistical physics, including zero range processes [25][26][27][28] (as a non-equilibrium steady state), mass transport [29][30][31], random trees [32,33], lattice models of quantum gravity [34][35][36], emergence of the longest interval in tied down renewal processes [37,38], wealth condensation [39] and diversity of Zipf's population [19].The system described by the model has a phase transition which is associated with a real-space condensation [23].
We will study the Rényi entropy for this model with a given micro-state being a particle distribution in the boxes σ = (s 1 , s 2 , ..., s N ) as described above.The Rényi entropy is defined as in Eq. ( 2) where p σ is the probability of the σ state: Z in the last equation is an abbreviation for Z S,N .With the Gibbs measure definition of the micro-states employed here the free-energy difference/Rényi entropy relations of (7,8) apply.This in turn allows us to relate the singular behavior of the Rényi entropy (density) to that of the free energy (density).
Our aim is to calculate H λ explicitly in the thermodynamic limit where r ∈ (0, 1), and then use this to obtain the singular behaviour, if it exists.The parameter r is a free parameter which is equal to the inverse particle density (i.e. the average number of particles per box).In the limit ( 14), the free energy F (β, r, S) is an extensive quantity which means that it grows linearly with the system size F (β, r, S) = Sf (β, r) + o(S) as S goes to infinity.We are interested in the coefficient f (β, r) of the leading term, which can be interpreted as the free energy per particle.Only if this coefficient is zero need the next-to-leading terms denoted by o(S) be considered.
In general, we will use convention that extensive quantities will be denoted by capital letters, while the corresponding densities by small-case letter.In particular, we will denote the Rényi entropy per particle (or the Rényi entropy density) by h λ .
Let us introduce a thermodynamic potential where "lim" in this equation means the thermodynamic limit as defined in (14).The function ϕ(β, r) gives the rate of exponential growth of the partition function with S in the thermodynamic limit (14), and it is directly related to the free energy density: ϕ(β, r) = −βf (β, r).Dividing both sides of ( 7) by S and taking the limit ( 14) we find a direct relationship between the Rényi entropy density and the thermodynamic potential ϕ For λ → ∞ the last equation reduces to h ∞ (β, r) = ϕ(β, r) while for λ = 0 to h 0 (β, r) = ϕ(0, r).Clearly, h 0 (β, r) is independent of β.It can be easily determined by enumeration of states which gives Substituting this into (15) we find in the thermodynamic limit ( 14) For the Shannon entropy (density) limit λ → 1 as can be seen by applying L'Hôpital's rule to (16).
The thermodynamic potential ϕ(β, r) can be found analytically using the saddle point method.The details can be found in [23,24,34] or in a more recent paper where results on the phase structure of the zeta-urn model are updated and collected in one place [40].Here we quote the result which is expressed in terms of a generating function: where Li β (z) is the polylogarithm: For α = 0 Two cases can be distinguished.For β ≤ 2, ϕ(β, r) can be expressed as parametric equations, where both ϕ and r are parameterized by α ∈ (0, ∞): and These equations are valid for all values from the range r ∈ (0, 1), which is an image of the range α ∈ (0, ∞) of the mapping (23).For β > 2, the image of the range α ∈ (0, ∞) is r ∈ (r c , 1) where r c = r c (β) is a critical value given by The saddle point solution (23,24) holds for r ∈ (r c , 1), while for r ∈ (0, r c ] the solution is linear in r For r ∈ (r c , 1) the system is in the fluid phase while r ∈ (0, r c ) in the condensed phase.In the condensed phase one box captures a finite fraction of all particles S as S → ∞ [23].This is a real-space condensation.It should be noted that in the condensed phase the Rényi entropy is determined by the bulk part of the distribution, which remains in the critical state, since the contribution from the condensate in a single box vanishes in the thermodynamic limit (14).
The parametric equations (23,24) can be used to plot the function ϕ(β, r) and thus also h λ (β, r) (16).We show two examples in Fig. 1 for β = 5/2 and β = 3/2 to illustrate the behaviour for the two cases mentioned above.The Rényi entropy density h λ (β, r) inherits its singularities from ϕ(β, r).The primary singularity is at r c (β) (25), but h λ (β, r) may also have a secondary singularity coming from ϕ(λβ, r) in (16) which is located at In this respect, the Rényi entropy density for the zeta-urn model is behaving (unsurprisingly, given (9,10,13)) as an equilibrium statistical mechanical system, with singularities at two different β values.It was found in [41] that this was not the case for the totally asymmetric exclusion process (TASEP), where H 2 was calculated by combinatorial means and found to possess no secondary singularities.It was suggested there that secondary singularities would generically be absent in such non-equilibrium systems, since they were a consequence of the relations (7,8), which are peculiar to equilibrium systems.Although the distribution of balls in a zeta-urn model can be considered as arising as a non-equilibrium steady state in a zero range process (ZRP) with suitable jumping rates for the balls [25], we are treating it as a purely equilibrium model here.
The function h λ (β, r) has a primary singularity at r c (25) for β > 2 and a secondary singularity at r c,λ (27) for λβ > 2, so there are four different cases:   The positions of the secondary and primary singularities merge for λ → 1 (the Shannon entropy).The behaviour is illustrated in Fig. 2. The primary singularities of h λ (β, r) at r c (25) are directly related to the singularities of ϕ(β, r).To determine the type of singularity of ϕ(β, r) at r = r c it is convenient to take the partial derivative of ϕ(β, r) with respect to r because the corresponding parametric equations for ∂ r ϕ(β, r) are simpler than those for ϕ(β, r) (23,24) and are therefore more useful in the analysis of critical point singularity.For r ∈ (0, r c ] we get while for r ∈ (r c , 1) and where α ∈ (0, ∞).The equations will be used as follows.First we will expand the right hand side of (29) to extract the dependence of α = α(r) on r, for r approaching r c from above.Then we will substitute α = α(r) into the expression on the right hand side of (30) to determine the type of singularity of ∂ r ϕ(β, r) for r → r c .To this end we will use the series expansion of the polylogarithm for a non-integer β [42]: For β ∈ (2, 3), ( 29) and ( 30) can be written as and with coefficients r c , µ c , a 1 and B > 0, depending on β.The dependence of the coefficients on β can be easily determined (for instance µ c (β) = log ζ(β)), but will not displayed in the analysis below, because we want to concentrate on the dependence on r.From the first equation we get 2) .Substituting this into the second equation leads to (35) with D = a 1 C.The coefficients µ c and D depend only on β, so if we take the derivative of both sides with respect to r we get with E = (β − 2)D being a positive function of β ∈ (2, 3).The second derivative ∂ 2 rr ϕ(β, r) is related to particle density fluctuations.For β ∈ (2, 3) the exponent changes from zero to infinity when β changes from 3 to 2, so the transition is of second or higher order.For β = 2 the transition disappears and there is no phase transition for β ≤ 2. On the other hand, for β = 3 the second derivative has a logarithmic singularity at r c .To see this, let us use the series expansion of the polylogarithm for an integer β [42] Li with H n = 1 + 1/2 + . . .+ 1/n being the n-th harmonic number.For β = 3, Eqs.( 29) and ( 30) take the form and where again the coefficients r c , b 1 , B, µ c and a 1 depend only on β.Calculating α as a function of r from the first equation we get with Substituting this into the second equation we get where d 1 = a 1 c 1 and D = a 1 C.As a consequence the second derivative has a logarithmic singularity for r → r What is essential in the last equation is that the second derivative diverges logarithmically when r → r + c .This means that for β = 3 the particle density fluctuations are infinite at the critical point: r → r + c .For β > 3, Eqs.( 29) and ( 30) take the form and and hence lim On the other hand lim as follows from (26).Hence the second derivative has a finite discontinuity for β > 3. Additionally, we see that the first derivative contains a singular term ∼ (r − r c ) β−2 , and therefore the second derivative contains a term ∼ (r − r c ) β−3 which makes higher derivatives diverge for r → r + c .For an integer β this term is ∼ (r − r c ) β−3 log(r − r c ), as follows from (38).To summarize, the second derivative ∂ 2 rr ϕ(β, r) has a power-law singularity with the exponent x (37) for β ∈ (2, 3), a logarithmic singularity (43) for β = 3 and a finite discontinuity for β > 3 at the critical point.For β > 3 it also contains a singular term ∼ (r − r c ) β−3 .
The singularity of ∂ 2 rr ϕ(β, r) at r c is inherited by the Rényi entropy density ( 16) This is the primary singularity.Additionally, the Rényi entropy can acquire a secondary singularity at r c,λ (27) when λβ > 2. The singularity type is the same as for the primary singularity, except that it appears at λβ rather than β.
For λ = 0, h 0 (β, r) (18) is independent of β and has no singular points in the range r ∈ (0, 1).Another exception is λ → 1 because the resulting singularity of h 1 (β, r) comes from the merging of the primary and the secondary singularities.One can expect from Eq. ( 19) that the power-law singularities will acquire an extra logarithmic factor from the derivative of a power depending on β.For example, the second derivative has the following singularity The logarithm here is generated from the derivative in the second term in (19).

III. DISCUSSION
We have calculated analytically the Rényi entropy for the abundance vector defined by the Gibbs measure for a zeta-urn model.In the thermodynamic limit for a suitable choice of parameters the model has a (condensation) phase transition, which manifests as a singularity of the free energy at the critical point.The Rényi entropy also has a singularity at this point, but in addition to this the Rényi entropy can, depending on its order, display a secondary singularity at another point (unlike the archetypal non-equilibrium model, the TASEP, as demonstrated in [41]).The secondary singularity is a phantom of the original singularity but itself is not directly related to any critical behaviour in the system.The mechanism which leads to the occurrence of the secondary singularity is quite generic, following from (7,8), so such secondary singularities will occur in other statistical mechanical models with Gibbs weights and phase transitions.We stress, however, that these secondary singularities are rooted in the mathematical definition of the Rényi entropy rather than in physical behaviour of the system.Below we illustrate this by a discussion of the secondary singularities for the Rényi divergence, where they arise from a comparison of two systems at different temperatures, so as such they cannot be a physical property of either one of the systems individually.
The Rényi divergence of order λ is a generalisation of the Kullback-Leibler divergence [43] (which is reproduced from the expression above in the limit λ → 1).The Rényi divergence for two Gibbs distributions p = {p σ = e −β1Eσ /Z(β 1 ), σ = 1, . . ., N } and q = {q σ = e −β2Eσ /Z(β 2 ), σ = 1, . . ., N }, for the same statistical system at different temperatures which should be compared with (7).For convenience, we have replaced arguments of ∆ λ by β 1 and β 2 which uniquely identify the thermal distributions p and q.The divergence is proportional to the temperature difference and the heat capacity of the system For the zeta-urn model, in the thermodynamic limit (14), Eq. ( 52) yields We see that apart from the primary singularities at r Returning to the interpretation of the Rényi entropy density as a diversity measure, it is interesting to look at the behaviour of h λ as r is varied in Fig. 1.For a given β as r is decreased (i.e. as the density of particles is increased) h λ initially increases, reaching a maximum value at The values taken by ∂ r ϕ(β, r) in (55,56) will depend on whether a singularity has been encountered or not, giving log ζ(β) for r ∈ (0, r c ] and K β (α(r)) for r ∈ (r c , 1), with similar considerations for ∂ r ϕ(λβ, r).Whether the maximum value of h λ (β, r) is attained as r is decreased before a singularity is encountered will depend on both λ and β.For instance, when β = 5/2 we can see in Fig. 1 that the maximum of h λ occurs before any singularities are encountered when λ ≤ 1, whereas it may encounter the secondary singularity at r c,λ (β) for sufficiently large λ before reaching its maximum.The Shannon entropy, h 1 , attains a maximum value in the fluid phase and then decreases linearly with r into the condensed phase after encountering the primary singularity as the density of particles is increased.In the second example in Fig. 1, β = 3/2, there is no primary singularity but the secondary one exists for λ > 4/3 and can lie on either side of the maximum of h λ depending on the value of λ.
It is also clear that whatever the value of β the maximum value of h λ decreases from that of h 0 as λ is increased and shifts to larger r so both the value of the "maximum diversity" and the density at which it occurs depend on the value of λ chosen for a given β.Similarly, increasing β for a given λ decreases the maximum value of h λ and shifts it to larger r.The task of maximizing the diversity for a zeta-urn model in the ensemble we consider thus depends both on what we mean by the diversity, e.g the choice of λ, and what parameters we have under our control, e.g.r and/or β.