Topological vertex formalism with O5-plane

We propose new topological vertex formalism for Type IIB $(p,q)$ 5-brane web with an O5-plane. We apply our proposal to 5d $\mathcal{N}=1$ Sp(1) gauge theory with $N_f=0,1,8$ flavors to compute the topological string partition functions and check the agreement with the known results. Especially for the $N_f=8$ case, which corresponds to E-string theory on a circle, we obtain a new, yet simple, expression of the partition function with two Young diagram sum.


I. INTRODUCTION
The topological string partition function has played an important role in finding the BPS spectrum (e.g., Nekrasov partition function) of the supersymmetric gauge theories with eight supercharges. Given a generic toric Calabi-Yau geometry, the partition function can be systematically computed based on topological vertex formalism [1][2][3]. The correspondence [4] between the toric diagram and the (p, q) 5-brane web diagram allows one to implement the formalism on a (p, q) 5-brane web. Recently, a generalized version of 5-brane web diagram, corresponding to non-toric Calabi-Yau geometry, was introduced in [5], and it has been proposed [6][7][8] that topological vertex formalism is applicable even for such non-toric cases simply by tuning Kähler parameters in a proper way [9][10][11][12].
It is then natural to ask whether topological vertex formalism can be also applicable to the (p, q) 5-brane web diagram whose corresponding Calabi-Yau geometry is not necessarily clear [42]. In this paper, we discuss that it is possible at least for some cases. In particular, we consider a Type IIB (p, q) 5-brane web with an O5-plane describing five-dimensional(5d) N = 1 Sp(N ) gauge theory, as depicted in Fig. 1(a) [13,14]. 5d pure Sp(N ) theory has a Z 2 valued discrete theta angle, either θ = 0 or π [15,16]. In Fig. 1(a), however, the difference in the theta angles does not look manifest [13]. It becomes much more distinct when one uses a generalized flop transition developed in [14,17]. For instance, Fig.  1(b) is the web diagram for the discrete theta angle θ = 0 (for odd N ) after the flop transition is performed on Fig.  1(a), where (1, 1) and (1, −1) 5-branes are attached to an O5-plane in a specific way [43]. In this paper, we propose new rules for such configuration in addition to the conventional topological vertex formalism, and present a new method that enables one to compute the Nekrasov partition function for such gauge theory constructed with (p, q) 5-branes with an O5-plane.
It is worth noting that the web diagram in Fig. 1  corresponds to the parameter region where the gauge coupling square is negative, which is denoted in [18] as "past infinite coupling". In this region, the description as 5d Sp(N ) gauge theory breaks down. Instead, better description is given by 5d SU(N + 1) gauge theory with Chern-Simons level κ = N + 3. This duality is first proposed in [19] with N f flavors and κ = N + 3 − N f /2, and further elaborated in [17,20,21]. The map between the parameters of two gauge theories are proposed in [19,21] and it is checked that the Nekrasov partition functions for the dual theories are identical under this map up to analytic continuation. Due to the non-trivial Chern-Simons level, the Nekrasov partition function for this 5d SU(N + 1) gauge theory has been less understood, although there are several partial results including [19,21]. Throughout the paper, we demonstrate our topological vertex formalism with an O5-plane would provide an effective way for computing Nekrasov partition function of this class. To this end, we focus on the simplest case, N = 1, as an instructive example of our proposal. Since Sp(1) = SU(2), the map gives symmetry of the partition function, which in fact corresponds to a Weyl transformation of the enhanced E N f +1 global symmetry [21].
The organization of the paper is as follows: In section II, we propose a new rule of topological vertex formalism for the intersection between 5-branes and an O5-plane appearing in Fig. 1(b) as well as still another type of intersection which also naturally appears through generalized flop transition. In section III, we compute (unrefined) Nekrasov partition function for SU(2) N f = 0, 1 and N f = 8 flavors based on the new formalism introduced in section II, and compare the obtained result with arXiv:1709.01928v3 [hep-th] 29 Apr 2020 the known one. In particular, the case with N f = 8 flavors corresponds to the E-string partition function. We then conclude with summary and future directions.

II. FORMALISM
In this section, we propose new rules for computing topological string partition function that involves a brane configuration with an O5-plane, in addition to the conventional (unrefined) topological vertex formalism. For notations, we follow [22] mostly.
According to the topological vertex formalism [1], the topological string partition function can be computed systematically based on the (p, q) 5-brane web diagram as follows: First, we assign different Young diagrams λ, µ, ν, · · · to different edges in the web diagram. Then, we introduce the edge factor (−Q) |λ| f λ n to each edge, where λ is the Young diagram assigned to the considered edge, and the vertex factor C λµν is assigned to each vertex, where λ, µ, ν are the Young diagrams assigned to the three edges sharing the vertex we consider. The Young diagrams in the edge factor are ordered in a clockwise way [44]. Here, Q is the exponentiated length of the 5branes, which corresponds to the Kähler parameter of the corresponding toric Calabi-Yau geometry. The framing factor f λ [45] is a specific function of g = e −β , with being the self-dual Ω-deformation parameter ≡ 1 = − 2 . The power n is fixed by the (p, q) charge of the adjacent 5-branes. See, e.g. [22] for detailed explicit expressions. Finally, C λµν is the (unrefined) topological vertex defined in [1], which is a specific function of g written in terms of the skew Schur functions [46]. Then the topological string partition function can be computed by multiplying these factors and summing over all possible Young diagrams as When we have an O5-plane, we need new rules for the part where 5-branes are intersected with an O5-plane. Let us consider a configuration where the (p, −1) 5-brane and (−p, −1) 5-brane intersect on the O5 − -plane as depicted in Fig. 2, where p is either p = 0 or p = 1. The configurations for both p = 0 and p = 1 naturally appear as a consequence of the generalized flop transition discussed in [14] [47]. Note that the case p = 0 corresponds to the two coincident NS5-branes attached to an O5-plane. The Kähler parameters associated with the (p, −1) and (−p, −1) 5-branes are Q 1 and Q 2 , respectively.
For a (p, q) 5-brane web with an O5-plane like Fig. 2, we now introduce the following new rule for the topological vertex computations: • Assign identical Young diagram Y to both the (p, −1) 5-brane and the (−p, −1) 5-brane as in Fig.  2(a).
• Introduce the new type of edge factor, for the configuration including the edges corresponding to (p, −1) and (−p, −1) 5-branes, where Equipped with these two new rules, we claim that the topological string partition function can be computed in the same way as in (1) even if the 5-brane web diagram includes O5-planes. These new rules would be more intuitive when we see the brane configuration from the the point of view of the covering space which includes the reflected images due to an O5-plane. Namely, the configuration in Fig. 2(a) can be naturally connect to the reflected 5-branes as shown in Fig. 2(b). The resulting configuration then look a single edge, and it is therefore natural to assign the identical Young diagram Y . The new edge factor in (2) is also analogous to the conventional edge factor.
We note, however, that the direction of arrows on the edges are all inverted compared with naive expectation. Also the + sign appearing in (2) is different from the one appearing in the conventional edge factor. Furthermore, +1 is added to the power of the framing factor compared with the naive expectation from Fig. 2 All these subtle differences appear in the following reason: If we compare the contribution coming from the subdiagram reflected due to an O5-plane to the original one, we find that the order of Young diagrams in the topological vertices should be reverse since the clockwise ordering is converted to a counter-clockwise ordering when we reflect the diagram along the O5-plane. From the identity, we find that the reversal of the order of the Young diagram is translated into the transposition of the Young diagram, which is equivalent to changing the direction of the arrow. Also, the prefactors in (4) account for the + sign in (2) as well as +1 in (3). This reflection technique is useful for practical computations. ? . 3: (a) N f = 0 case where physical parameters are expressed in accordance with the SU(2) parametrization: QF = A 2 accounts the Coulomb branch modulus, while QB 2 QF = q accounts for the instanton factor, and they satisfy A partial brane configuration connected to the reflected image by an O5-plane.

III. COMPUTATION
In this section, we apply our proposal for the topological vertex method for a brane configuration with an O5-plane to a few specific well known examples: 5d Sp (1) theory with N f = 0, 1 and N f = 8 flavors. More specifically, we compute the BPS partition function of the 5d Sp(1) theory and test our method by comparing our obtained result with the well-known SU(2) theory result.
A. N f = 0 case Consider 5d pure Sp(1) theory with the discrete theta angle θ = 0 (mod 2π) whose brane configuration is given in Fig. 3 [14]. Following the computation procedure in section II based on Fig. 3(a), we obtain the partition function as where we glued the red strip and the green strip in Fig.  3, defined as Instead of computing (5) directly, we implement the reflection technique introduced in the previous section, which is more convenient and systematic. First we use the identity (4) to re-express the red and green strips as a single strip as in Fig. 3(b), which involves the summation over λ in (5). We denote the resultant strip by Z strip µ1µ2 . The full partition function is then written as Notice that Z strip µ1µ2 is nothing but a conventional strip diagram contribution which is already discussed in [23], and it is straightforward to compute Here, the Kähler parameters in Fig. 3 are related to the Coulomb modulus A = e −βa and the instanton factor q as follows, We then expand the partition function (8) in terms of q to compare it with the known results [22,24]. We checked their agreement up to 10 instanton orders.
Adding a flavor D5-brane to the brane configuration is straightforward. As a representing example, we discuss the N f = 1 case based on Fig. 4, which includes the configuration in Fig. 2 with p = 0. We note that depending on the region of the mass parameter, one can also use a diagram with the configuration in Fig. 2 with p = 1. Although the cases of p = 0 and p = 1 may look different at first sight, one can easily check that either case gives the same strip when we separate and glue the half of the diagram to obtain Fig. 4(b) [14]. The only difference is either the flavor brane is placed above or below the position of the O5-plane, which does not change the computation at all.
Repeating the procedure explained in the previous subsection, one readily obtains the partition function for the N f = 1 case, where we used a shorthand nation for Kähler parameters, The relation between the Kähler parameters and the gauge theory parameters is given as follows: where A is the Coulomb modulus, M = e −βm is the mass parameter, and q is the instanton factor. Again, by expanding (13) in terms of q, we checked our method. Our partition function (13) agrees with the known result [22,24] up to 10 instantons, where we used the flop transitions in the perturbative part. In a similar fashion, the partition functions for the N f = 2, · · · , 7 cases can be straightforwardly computed.
C. N f = 8 case: E-string theory We now consider the N f = 8 case which would serve as a nontrivial test for our method. 5d SU (2) theory with N f = 8 flavors is special in the sense that its UV fixed point exists in 6d. It is in fact 6d E-string theory compactified on a circle whose partition function or elliptic genus was recently computed in [25][26][27][28]. It is known that the N f = 8 case can be realized by two fractional NS5-branes on an O6 − -plane in Type IIA setup, whose T-dual picture is Type IIB (p, q) 5-brane web with two O5-planes as depicted in Fig. 5.
As two O5-planes are required in Fig. 5, the covering space for (p, q) 5-brane with O5-planes is periodic as shown in Fig. 5, where the periodicity is given by the instanton factor squared q 2 . The periodic strip diagram appearing in Fig. 5 is exactly the one computed in the context of M-string [29][30][31][32] but with specific tuning of the Kähler parameters due to the O5-planes. For the periodic strip, we replace R µν (Q) by its infinite product which yields the topological string partition function for 5d SU(2) gauge theory with N f = 8 flavors , where x I , y i are the labels of the location on the (p, q) web diagram with two O5-planes in Fig. 5. Here, we used the analytic continuation corresponding to the flop transition: The parametrization is given by where M i = e −βmi are eight mass parameters and Λ ≡ q . We have dropped the factors like Θ ∅∅ (y ±1 i y ±1 j ), which do not depend on the Coulomb modulus A, as they are part of the "extra factor" [6,22,25,33]. Taking into account the extra factors as well as allowing the flop transitions in the perturbative part [50], we found that our result (16) is in agreement with the known partition function [28]. Our proposal hence provides a new, yet simple, expression for R 4 × T 2 partition function of 6d E-string theory.

IV. CONCLUSIONS AND DISCUSSIONS
In this paper, we proposed a way to implement the topological vertex formulation for (p, q) 5-brane configurations with an O5-plane. The key idea is that, based on a special phase of the brane configuration where 5-branes stuck on an O5-plane meet at a point on the O5-plane, we assign an identical Young diagram to two such 5-branes and introduce the new edge factor (2) corresponding to such 5-brane configuration.
To test our proposal, we considered 5-brane webs for the 5d Sp (1) theory with N f = 0, 1 flavors, compared the partition functions computed based on our proposal with the known SU(2) partition functions. For each case, we checked these two partition functions by expanding them in terms of the instanton factor, and found that they do agree up to 10-instanton order. Another nontrivial check we did is 5d Sp(1) theory with N f = 8 flavors. As it is 6d E-string theory on a circle, the brane configuration consists of two O5-planes and hence naturally shows periodic structure. We found that our partition function for the N f = 8 case also agrees with the known E-string elliptic genus partition function.
It is feasible to apply our method to higher rank Sp(N ) gauge theory which then gives rise to the dual SU(N + 1) theory, and, hence, confirm the duality [19] between two theories in a more manifest way.
Our formalism is also valid even for the S-dual descriptions of (p, q) 5-brane configuration with an O5plane, which lead to the 5-brane web with an ON-plane [13,17,34]. In particular, the web diagram proposed in [17] as the "microscopic" description of an ON 0 -plane, which is used to construct D-type quiver gauge theories, is exactly the S-dual of the case p = 0 in Fig. 2. Therefore, it should be straightforward to reproduce the (unrefined) Nekrasov partition function for D-type quiver gauge theories using our proposal [51].
Our proposal may enable one to compute various topo-logical string partition computations where the conventional topological vertex method is not applicable, such as 5d SO(M ) gauge theories with hypermultiplets in vector as well as spinor representations [40] based on the web diagram proposed in [13]. Finally, it would be useful to extend our method to the refined topological vertex, as the computations in this paper are done based on an unrefined version of topological vertex formulation.
[43] For the discrete theta angle θ = π, one obtains a different brane configuration after the flop transition. For details, see [14].
[45] The explicit form of the framing factor associated with an edge is given by λi for Young diagram λ = (λ1, λ2, · · · , λ (λ) ) assigned to the edge, and The explicit expression is given by and s λ/η (x) is a skew-Schur function.
[49] We note that the parametrization (18) is the same parametrization discussed in [21], accounting the parameter map between 5d Sp(N ) gauge theory and SU(N + 1) gauge theory [19].
[50] For comparison, we first rewrite (16) in a plethystic exponential form, whose exponent is given as the expansion in terms of the instanton factor q. The q 0 term in this expansion is identified as the perturbative part. In order to see the agreement of the perturbative part, we need to change the Kähler parameters in some of the terms as Q → Q −1 by hand, which is interpreted as the flop transition. The A 0 term in the exponent is called "extra factor" and we remove such contributions. This is the same technique used e.g. in [21,28].
[51] In preparing this paper, we became aware that the partition function for D-type quiver gauge theories is considered in [41] based on a similar setup.