Functional renormalization group approach and gauge dependence in gravity theories

We investigate the gauge symmetry and gauge fixing dependence properties of the effective average action for quantum gravity models of general form. Using the background field formalism and the standard BRST-based arguments, one can establish the special class of regulator functions that preserves the background field symmetry of the effective average action. Unfortunately, regardless the gauge symmetry is preserved at the quantum level, the non-invariance of the regulator action under the global BRST transformations leads to the gauge fixing dependence even under the use of the on-shell conditions.


Introduction
The interest to the non-perturbative formulation in quantum gravity has two strong motivations. First, there are a long-standing expectations that even the perturbatively nonrenormalizable models such as the simplest quantum gravity based on general relativity may be quantum mechanically consistent due to the asymptotic safety scenario [1] (see [2,3] for comprehensive reviews). On the other hand, there is a possibility that the nonperturbative effects may provide unitarity in the fourth derivative theory by transforming the massive unphysical pole, which spoils the spectrum of this renormalizable theory [4]. The presence of such a massive ghost violates the stability of classical solutions (see e.g. discussion and further references in [5]). At the quantum level, the perturbative information is insufficient to conclude whether in the dressed propagator the ghost pole does transform into a non-offensive pair of complex conjugate poles [6].
The perturbative renormalization group in this model is well-explored [7,8,9], but it is not conclusive for the discussion of the dressed propagator. In general, the attempts to explore this possibility in the semiclassical and perturbative quantum level [10,11] has been proved non-conclusive [12], and hence the main hope is related to the non-perturbative calculations in the framework of Functional Renormalization Group approach [13] (see [14] for an extensive review).
Thus, in both cases the consistency of the results obtained within Functional Renormalization Group approach is of the utmost importance. In this respect there are two main dangers. For the quantum gravity models based on general relativity, the running of Newton constant in four-dimensional spacetime is always obtained on the basis of quadratic divergences. These divergences are known to have strong regularization dependence. In particular, they are absent in dimensional regularization and can be freely modified in all known cut-off schemes by changing the regularization parameter. This part of the problem does not exist for the Functional Renormalization Group applied to the fourth derivative quantum gravity. However, in this case there is yet another serious problem, related to the gauge-fixing dependence of the effective average action. This problem is the subject of the present work. In the previous publications [15,16,17] we explored the gauge fixing dependence in Yang-Mills theories and it was shown that such dependence for the effective average action does not vanish on-shell, except in the fixed point where this object becomes identical with the usual effective action in quantum field theory. Except this special point there is uncontrollable dependence on the set of arbitrary gauge fixing parameters, and thus one can expect a strong arbitrariness in the renormalization group flows which lead to the fixed point and, in fact, define its position and proper existence. The main purpose of the present work is to extend this conclusion to quantum gravity. It is remarkable that one can complete this task for the quantum gravity theory of an arbitrary form, without using the concrete form of the action. One can use this consideration, e.g. for the superrenormalizable models of quantum gravity, when the perturbative renormalization group may be exact and, moreover, completely independent on the gauge fixing [18,19]. This example is somehow the most explicit one, since it shows that the transition from standard quantum field theory to the functional renormalization group (FRG) may actually spoil the "perfect" situation, namely exact and universal renormalization group flow.
In Yang-Mills theories [20], the gauge symmetry of the initial action is broken on quantum level due to the gauge fixing procedure in the process of quantization. In turn, the effective potential depends on gauge [21,22,23,24]. This dependence occur in a special way, such that it disappears on-shell [25,26], which means that it is possible to give physical interpretation to the results obtained at the quantum level.
One of the well-developed non-perturbative methods in Quantum Field Theory to study quantum properties of physical models beyond the perturbation theory is the FRG approach [27,28] (see also the review papers [29,30,31,32,33,34]). As we have already mentioned above, when applied to gauge theories, this method leads to obstacles related to the on-shell gauge dependence of the effective average action.
There are some efforts to solve this problems. One of them consists from reformulation of Yang-Mills theory with the application of a gauge-invariant cut-off dependent regulator function that is introduced as a covariant form factor into the action of Yang-Mills fields, which leads to an invariant regulator action [35,36]. As a consequence, the effective average action is gauge invariant on-shell. A second approach [37,38] is based in the use of the Vilkovisky-DeWitt covariant effective action [39,40]. This technique provides gauge independence even off-shell, but it introduce other types of ambiguities. An alternative formulation was presented in [15], it consists of an alternative way of introducing the regulator function as a composite operator. When applied in gauge theories, this approach leads to the on-shell gauge invariance of the effective average action.
In the present work, we apply the background field method [41,42,43] (recent advances for the gauge theories can be found in [44,45,46,16,47,48,49] and discussion for the quantum gravities case in [50]) in the FRG approach as a reformulation to the quantization procedure for quantum gravity theories to study the gauge dependence problem in this kind of theory. This method allows us to work with the effective action which is invariant under the gauge transformations of the background fields.
Despite the numerous aspects of quantum properties successfully studied with the background field method, [51,52,53,54,55,56,57,58,59,60], the gauge dependence problem remains important [16] and need to be considered in more details. We obtain restrictions on the tensor structure of the regulator functions which allows us to construct a gauge invariant effective average action. Nevertheless, the effective average action remains dependent of the gauge choice at on-shell level.
The paper is organized as follows. In sectionsect2 we introduce general considerations of quantum gravity theories through the background field method. In sectionsect3 we consider the FRG approach for quantum gravity theories and find conditions that we must impose in the regulator functions to maintain the gauge invariance of background effective average action. Based on this, we also present some interesting candidates to the regulator functions. In sectionsect4 we prove the gauge dependence of vacuum functional for the model under consideration. Finally, our conclusions and remarks are presented in section sect5.
In the paper, DeWitt notations [61] are used. The short notation for integration in D dimension is d D x = dx. All the derivatives with respect to fields are left derivative unless otherwise stated. The Grassmann parity of a quantity A is denoted as ε(A).

Quantum gravity in the background field formalism
Let us consider an arbitrary action S 0 = S 0 (g), where g = g αβ (x) is the metric tensor of an arbitrary Riemann manifolds. We assume that the action is invariant under general coordinates transformations which leads to the metric transformations and consider the infinitesimal form of these transformations, x ′σ = x σ + ξ σ (x), when The diffeomorphism (3) can be considered as the gauge transformation for g αβ (x) where are the generators of the gauge transformations of the metric tensor, and ξ σ (y) are the gauge parameters. The algebra of generators is closed, namely where with F λ µν (x, y, z) = −F λ νµ (x, z, y) (8) are structure functions of the algebra which does not depend on the metric tensor. Let us stress that the mentioned features are valid for an arbitrary action of gravity, since the algebra presented above is independent on the initial action. Therefore, any theory of gravity is a gauge theory and the structure functions are independent of the fields, that means quantum gravity is similar to the Yang-Mills theory.
An useful procedure to quantize gauge theories is the so called background field formalism. In what follows, we shall perform the quantization of gravity on an arbitrary external background metricḡ αβ (x). The standard references on the background field formalism in quantum field theory are [41,42,43] (see also recent advances for the gauge theories in [16] and [48] for the discussion of quantum gravity).
In the background field formalism, the metric tensor g αβ (x) is replaced by the sum whereḡ αβ (x) is an external (background) metric field and h αβ (x) is the variable of integration, also called quantum metric. Thus, the initial action is replaced by The Faddeev-Popov S F P (φ,ḡ) action is constructed in the standard way [62] where S gh (φ,ḡ) is the ghost action and S gf (φ,ḡ) is the gauge fixing action. In the presence of external metricḡ αβ (x), they can be written as where and χ α (x;ḡ, h) are the gauge fixing functions, are the ghost and anti-ghost fields, respectively, and B α (x) are the Nakanishi-Lautrup auxiliary fields. The Grassmann parity of all quantum fields are as follows, The ghost numbers are Independently of gauge fixing function choice, the action (10) is invariant under a global supersymmetry transformation, known as BRST symmetry [63,64]. The gravitational BRST transformations were introduced in [65,4,66] and can be presented as where λ is a constant Grassmann parameter. In condensed notation, we can write the BRST transformations as where The generating functional of Green functions is defined as where W (J,ḡ) is the generating functional of connected Green functions. In Eq.(17) the product of the sources J i (x) and the fields φ i (x) was written in the condensed notation of DeWitt [61]. Explicitly, with the Grassmann parities ε(J i ) = ε(φ i ) and ghost numbers gh(J i ) = gh(φ i ).
The effective action Γ(Φ,ḡ) is defined in terms of the Legendre transformation where Φ = {Φ i } are the mean fields and the J i are the solution of the equation It is well-known [25,26] that the effective action is gauge independent on-shell, At this moment we have considered only the transformations of the quantum fields. However, the background metric also transform together with the quantum fields in the socalled background field transformations. The rules of such transformation can be written, in the local formulation, as where ω σ = ω σ (x) are arbitrary functions. The background field transformations have the same structure of tensor transformations for tensors of types (0, 2) and (1, 0). The background invariance of Faddeev-Popov action for quantum gravity is known [48] and reads A consequence of (23) is the gauge invariance of (17). Namely, From this expression it is possible to prove that Γ(Φ,ḡ) is also gauge invariant In the next sections we will discuss this and other features for quantum gravity theories in the framework of the FRG approach.

FRG approach for quantum gravity theories
The main idea of functional renormalization group (FRG) is to use instead of Γ an effective average action, Γ k , where k is a momentum-shell parameter [27], in a way that In order to obtain Γ k (φ,ḡ), we introduce the average action where S k (φ,ḡ) is the scale-dependent regulator action defined in curved spacetime and the Lagrangian density is written as where the regulator functions R kαβ (x;ḡ) = 0, which means that the average action recovers the Faddeev-Poppov action (10) in the limit when k → 0. The regulator functions R (1)αβγδ k (x;ḡ) also obey, by construction, the symmetry properties We want to solve the problem of average action invariance under background field transformations, namely where the relation (23) is used.
In what follows, we present explicit calculation of variation of action (28). For this purpose, we write (32) as For the first term in (33) we have where integration by parts was used. The variation of L k (φ,ḡ) in second term of equation (33) can be presented as In terms of transformations (22), the above expression reads Thus, by means of Eqs. (34) and (36) the variation of the action can be written in the compact way where kαβ (x;ḡ) + R (2) kσβ (x;ḡ)∂ α ω σ + R kασ (x;ḡ)∂ β ω σ . (38) In order to ensure the invariance of (28), it is necessary that the following conditions are satisfied ωkαβ (x;ḡ) = 0.
As a result, we obtain expressions for the variation of the regulator functions, and If the relations (40) and (41) are fulfilled, then the action S k (φ,ḡ) is invariant under background field transformations. Therefore, the regulator functions should have a tensor structure in order to ensure the invariance. Thus, taking into account the symmetry properties presented in (31) we can propose the following solutions for the regulator functions and where R (1,2) k (¯ ) and Q k (¯ ) are scalar functions and¯ is the d'Alembertian operator defined in terms of the covariant derivatives of the background metric: with the metricity property∇ It is possible to show that (42) and (43) presents the same variational structure of (40) and (41), respectively. By using the inverse background metric variation we have The derivatives in metric tensor and in functions R (1) k (¯ ) and Q k (¯ ) can be combined to obtain As a result, it is possible to see that (40) is satisfied. For the second function its variation can be expressed, after some algebra, as The combination of derivatives in metric tensor and in scalar function leads to which has the same structure as (41). Finally, the scale-dependent regulator Lagrangian density (29) in terms of (42) and (43) reads which maintains the background field symmetry, δ ω S k (φ,ḡ) = 0.

Gauge dependence of effective average action
In order to understand the gauge invariance and gauge dependence problems in the background field method, we shall consider the generating functionals of Green functions where is the fermionic gauge fixing functional and is the generator of BRST transformations (15). As far as we saw in the previous section, the regulator action (28) does not depend on the gauge Ψ(φ,ḡ). Now, we shall consider another choice of gauge fixing functional Ψ → Ψ + δΨ and set J = 0 in (52). Thus, where We will try to compensate the additional termRδΨ in (55). To do this, we change the variables in the functional integral related to the symmetries of action S F P (φ,ḡ), namely the BRST symmetry and the background gauge invariance. First, we shall consider the BRST symmetry (14), but trading the constant parameter λ by a functional Λ = Λ(φ,ḡ). The variation of (28) under such transformation is the following where kαβ (x;ḡ)C β (x) +C α (x)R (2) kαβ (x;ḡ)δ B C β (x) After some algebra, δ B L k (φ,ḡ) reads as From the above expression, it is clear that the action S k (φ,ḡ) is not invariant under BRST transformations δ B S k (φ,ḡ) = 0. The Jacobian J 1 of such transformation can be obtained in the standard way where the functional derivatives are It is possible to choose a regularization scheme such that δ(0) = 0. As a result, the Jacobian for BRST transformations is It is also interesting to consider the background gauge transformation related to expressions (22). As far as the regulator functions transform as (40) and (41), the action S k (φ,ḡ) is invariant under such transformation. But now, instead of functions ω = ω σ (x) we shall consider the functional Ω σ = Ω σ (x, φ,ḡ). The action (28) remains invariant, and the corresponding Jacobian of this transformation can be obtained as before with the following functional derivatives: As before, δ(0) = 0. Thus, the Jacobian for background gauge transformations reads If it is possible to fulfill the condition the generating vacuum functional Z kΨ (ḡ) does not depend on the gauge fixing functional Ψ. In order to verify that, let us expand the functionals Λ and Ω, with Grassmann parity ε(Λ) = 1 and ε(Ω) = 0 and ghost numbers gh(Λ) = −1 and gh(Ω) = 0 in the lower power of ghost fields Λ = Λ (1) + Λ (3) , where The terms that vanish in (68) and do not depend on ghost fields, leads to Analyzing the terms which are linear in the anti-ghost fields, which contains the auxiliary fields B(x), we obtain and where Now, the vanishing terms with structureC(x)C(x) can be related to the second order of Ω σ (x, h,ḡ) functional leading to a differential equation for ω σ (2) αβ (x, h,ḡ), From Eq. (74) since Ω σ(0) (x, h,ḡ) is an arbitrary function we can not have just one particular solution. In addition, the λ (3) αβγ relation in (76) creates non-local term of structure BCCCC which can be only eliminated if we consider new functional Λ of higher orders in ghost fields. Even so, this process would repeat endlessly. The only case left for us is to consider the simple solution when Ω σ = 0 and Λ = Λ (1) we have the result Z kΨ+δΨ (ḡ) = dφ exp i [S kF P (φ,ḡ) + δS kF P (φ)] , Z kΨ (ḡ) = Z kΨ+δΨ (ḡ) .
As final result, the vacuum functional in the FRG approach for gravity theories depends on the gauge fixing even on-shell, which leads to a gauge dependent S-matrix.

Conclusions and Perspectives
We explored the problem of gauge invariance and the gauge-fixing dependence using the background field formalism, for gravity theories in the FRG framework. It was shown that the background field invariance is achieved when the regulator functions are choosen to have the tensor structure. Nevertheless, even in this case the on-shell gauge dependence cannot be cured in the standard FRG approach which we dealt with. In this respect the situation is qualitatively the same as in the Yang-Mills theories, as it was discussed in [15].
The on-shell gauge dependence takes place due to the fact that the regulator action (28) is not BRST-invariant. It turns out that this is a fundamentally important feature, that can be changed only by trading the standard and conventional FRG framework to an alternative one, which is based on the use of composite operators for constructing the regulator action. Unfortunately, until now there is no way to perform practical calculations in this alternative formulation. For this reason, taking our present results into account, it remains unclear whether the quantum gravity results obtained within the FRG formalism can have a reasonable physical interpretation. One can expect that all predictions of this formalism will depend on an arbitrary choice of the gauge fixing. Thus, one can, in principle, provide any desirable result, but the value of this output is not clear. Alternatively, there should be found some physical reason to claim that one special gauge fixing is "correct" or "preferred" for some reason, but at the moment it is unclear how this reason can look like, since the original theory is gauge (diffeomorphism) invariant.
The results of the considerations which we described above make more interesting the discussion of the possible ways to solve the problem of on-shell gauge fixing dependence, such as the ones suggested in [35,36,34] or in [15]. In our opinion, the last approach is more transparent and physically motivated, but (as we have already mentioned above) there is no well-developed technique of using it for practical calculations.