Maximum Entropy Distributions of Scale-Invariant Processes

Organizations of many variables in nature such as soil moisture and topography exhibit patterns with no dominant scales. The maximum entropy (ME) principle is proposed to show how these variables can be statistically described using their scale-invariant properties and geometric mean. The ME principle predicts with great simplicity the probability distribution of a scale-invariant process in terms of macroscopic observables. The ME principle offers a universal and unified framework for characterizing such multiscaling processes.

Self-similarity is one of the common statistical properties of many natural phenomena encompassing a wide range of space or time scales. Scale invariance is a form of self-similarity, where these properties are scaleindependent. The concept was first introduced in the context of turbulence by Kolmogorov [1], who used probabilistic tools to describe the variability of the moments of velocity increments across a wide range of spatial scales. Success of this type of analysis was experimentally confirmed years later [2][3][4]. Yet the scaling properties were not analyzed locally until recent years by detecting spatial transitions in some satellite-derived scalar variables over extensive domains [5][6][7][8][9]. One of the outstanding issues is the theoretical prediction of the statistical behavior based on scale-invariant properties. The question is: would it be possible to use the known multiscaling moments to determine probability distributions? This study considers a possible approach, i.e., the principle of maximum entropy (ME), to answer that question.
First developed for statistical mechanics by Jaynes [10], the ME theory has been successfully applied to many scientific and engineering problems [11][12][13]. Its application in hydrology started in the early 1960s [14] and has attracted growing attention [15][16][17][18][19][20]. In this study we attempt to use the ME theory to characterize some important variables in hydrometeorology: the drainage area of a river network, soil moisture, and topography.
As an inference algorithm based on Bayesian probability theory, the ME theory selects the ''most probable'' microscopic configuration of a system among all possibilities consistent with macroscopic observables. According to the ME theory, the probability representing the current state of knowledge is the one associated to the maximum information entropy. The information entropy is a measure of uncertainty or lack of information about a physical system [21]. The generality and power of ME is rooted in the fact that the Bayesian interpretation of probability allows probability theory to be applicable to any situation where statistical inference based on incomplete information is sought. Although it is not a physical law per se, the ME theory has physical significance when applied to natural processes.
The ME formalism may be summarized as follows. The Shannon information entropy S I [22] of a probability density function pðxÞ of a continuous variable x is defined as pðxÞ log pðxÞ mðxÞ dx: Consider that some macroscopic properties of a system are expressed in terms of constraints in the form, where f k is an arbitrary function of x, F k the given parameters, and mðxÞ a given prior distribution. Maximizing S I in Eq. (1) subject to the constraints of Eq. (2) leads to a (normalized) probability density function, where k is the Lagrangian multiplier associated with the constraint F k through, and Z is the well-known partition function, pðxÞ in Eq. (3) is referred to as the ''ME distribution'' herein.
Next, we demonstrate that the distributions of some scale-invariant variables can be derived using the ME formalism. Here we present two cases: (1) drainage area of river network (DARN), and (2) surface soil moisture field (SSM) and topography as captured in a digital elevation map (DEM).
(1) DARN.-We denote the drainage area [16] by a with A 1 < a < A 2 with A 1 and A 2 as the upper and lower limit of a, respectively. It is straightforward to show that the ME distribution of a, according to Eq. (3), subject to the constraint of a given geometric mean, A g , assuming mðaÞ ¼ 1, is the power-law distribution [23], where the Lagrangian multiplier is related to the constraint A g through, For the case of A 2 ) A 1 , Eq. (8) leads to an explicit function of in terms of A g , ; (9) depending on whether > 1 or < 1.
The ME predicted power-law exponent has been validated using observations of drainage areas derived from a DEM map of the Grand Canyon from the U.S. Geological Survey (USGS) data archive. The ME predicted ¼ 1:32, according to Eq. (9), is in close agreement with ¼ 1:30 obtained by direct curve fitting of the empirical probability distribution of a according to Eq. (7). Equation (9) has also been tested for the river networks of Puerto Rico using the USGS DEM data with similar results (not shown). The close agreement between predicted and observed power-law exponents reveals the link between statistics and the macroscopic properties of a physical system.
(2) SSM and DEM.-We denote a SSM field or a DEM by zðxÞ wherex is a two-dimensional location vector, and pðz 1 ; z 2 Þ the joint distribution of z 1 zðx 1 Þ and z 2 zðx 2 Þ. Consider the following constraints: (i) Given multiscaling moments of the incremental process jz 1 À z 2 j, ZZ pðz 1 ; z 2 Þjz 1 À z 2 j q dz 1 dz 2 ¼ a q r ðqÞ ; 1 q M; where r jx 1 Àx 2 j is the separation distance, ðqÞ refers to the multiscaling exponent independent of r, and a q is a constant parameter. (ii) Given the geometric mean of jz 1 À z 2 j, where z i is observed value of z at locationx i and n is the number of sampling locations. These constraints are based on some previous studies showing that variables such as drainage area, discharge and energy release, soil moisture, increments of topography, etc., are self-similar characterized by the multiscaling moments or power-law distributions [16,[23][24][25][26][27].
The ME distribution according to Eq. (3), subject to constraints Eqs. (10) and (11) assuming mðz 1 ; z 2 Þ ¼ 1, is with where q is the Lagrangian multiplier associated with the qth moment constraint. q is related to ðqÞ according to Eq. (4), a q r ðqÞ ¼ @ logZ @ q ; 1 q M: For the case of M ¼ 1, Eq. (12) reduces to a Gamma distribution with Z expressed as an explicit function of 1 and 0 : Equations (14) and (15) lead to a q r ðqÞ ¼ 1 À 0 1 : The right-hand side of Eq. (16) is simply the expected value of a Gamma distribution.
For the case of M ¼ 2, no simple analytical solution of Z, as a function of 0 , 1 , and 2 , like that in Eq. (15) for the case of M ¼ 1, is available.
Two data products are used to validate the ME distributions Eqs. (12) and (16): the AMSR-E soil moisture L2B product and the USGS National Elevation Dataset (NED). The AMSR-E L2B soil moisture product provides estimates of soil moisture of the top 1 cm derived from brightness temperature at 10.7 GHz with resolution of 38 km, regridded into a global cylindrical 25 km Equal-Area Scalable Earth Grid. The NED raster elevation data with a resolution of one arc second (30 m) over the continental U.S.A. are derived from multiple satellite sensors. NAD83 and NAV88 are consistently used as horizontal and vertical datums, respectively, and all the data are regridded using a geographic projection. Soil moisture accuracy (root-meansquare difference) is about 60 kg m À3 , which almost holds the typical incremental values of 100 kg m À3 . The vertical accuracy (absolute error) of the NED product is on the order of 7-15 m. Equations (12) and (16) have been tested over 15 regions. Results for four of them are shown in this paper and referred to as R1SSM and R2SSM in Fig. 1, and R1DEM and R2DEM in Fig. 2 Figures 1 and 2 compare the ME distributions of SSM and DEM increments, according to Eq. (12), with the empirical ones for different separation distances r. It is evident that the Gamma distribution for M ¼ 1 agrees closely with the data (log-log regression coefficients of $0:97 and 0.99 for SSM and DEM, respectively). However, the predicted ðqÞ according to Eq. (16) does not agree with the observed values when the ME distribution is given in Eq. (12) for most of the regions under study. Relative errors are from 0.9% to 32% for the case of SSM, and up to 94% for the case of DEM. This finding suggests that the first moment constraint alone is not sufficient to characterize the multiscaling processes such as SSM and DEM. It is the ME distribution for M ¼ 2 that is almost identical to the empirical distributions (compare top and bottom panels). Note that the value of Z in Eq. (12) has been also estimated by fitting the empirical values with log-log regression coefficients of around 0.99, as no analytical solution of Z is available. It is also worthwhile to mention that the data [shown in Figs. 1 and 2] are positively skewed (i.e., right skewed), implying that the logarithmic transformation of the processes would be appropriate in data analysis.  18,2009 and region R1SSM, associated empirical (P e ) and the ME distributions for M ¼ 1, 2 (P t;M¼1 and P t;M¼2 ) according to Eq. (12). Right panels: same for region R2SSM. Maps are represented in longitude and latitude degrees. All probabilities are plotted versus the absolute value of the increments Áz ¼ jzðx 1 Þ À zðx 2 Þj for different separation distances r ¼ jx 1 Àx 2 j. Fig. 1 for regions R1DEM and R2DEM, which corresponds to the 1 arc second NED data.

FIG. 2. Same as in
We also compared the ME distribution subject to constraint (i) only, Eq. (10), with the empirical distribution (figure not shown), It is found that the ME distribution, Eq. (17), for M ¼ 1 only fits the tails of the empirical distributions. Equation (17) for M ¼ 2, a Gaussian distribution, does not capture all features of the empirical ones either. It turns out that the combination of the constraints (i) and (ii) is crucial to yield the best agreement between the ME and empirical distributions for SSM and DEM fields. We conclude that the geometric mean contains crucial information about the multiscaling processes of SSM and DEM. As a result, the geometrical mean may offer important clues regarding the physical mechanisms underlying the observed statistical properties.
In summary, we have demonstrated that the ME theory allows the statistical behavior of some scale-invariant processes such as DARN, SSM, and DEM to be characterized by a small number of macroscopically observable quantities. Such scale-invariant behavior resulting from self-organization emerges as the most probable and macroscopically reproducible state. It turns out that the geometric mean provides essential information for shaping river networks. The geometric mean is also identified as an important parameter, in addition to the moments, in characterizing multiscaling incremental processes of soil moisture and topography. More importantly, this analysis supports the assertion that the ME theory is a universal and unified framework to characterize those processes governed by scale-invariant laws.