Observability and Controllability of Nonlinear Networks: The Role of Symmetry

Observability and controllability are essential concepts to the design of predictive observer models and feedback controllers of networked systems. For example, noncontrollable mathematical models of real systems have subspaces that influence model behavior, but cannot be controlled by an input. Such subspaces can be difficult to determine in complex nonlinear networks. Since almost all of the present theory was developed for linear networks without symmetries, here we present a numerical and group representational framework, to quantify the observability and controllability of nonlinear networks with explicit symmetries that shows the connection between symmetries and nonlinear measures of observability and controllability. We numerically observe and theoretically predict that not all symmetries have the same effect on network observation and control. Our analysis shows that the presence of symmetry in a network may decrease observability and controllability, although networks containing only rotational symmetries remain controllable and observable. These results alter our view of the nature of observability and controllability in complex networks, change our understanding of structural controllability, and affect the design of mathematical models to observe and control such networks.


I. INTRODUCTION
An observer model of a natural system has many useful applications in science and engineering, including understanding and predicting weather or controlling dynamics from robotics to neuronal systems [1].A fundamental question that arises when utilizing filters to estimate the future states of a system is how to choose a model and measurement function that faithfully captures the system dynamics and can predict future states [2,3].An observer is a model of a system or process that assimilates data from the natural system being modeled [4], and reconstructs unmeasured or inaccessible variables.In linear systems, the key concept to employ a well designed observer is observability, which quantifies whether there is sufficient information contained in the measurement to adequately reconstruct the full system dynamics [5,6].
An important problem when studying networks is how best to observe and control the entire network when only limited observation and control input nodes are available.In classic work, Lin [7] described the topologies of graph directed linear networks that were structurally controllable.Incorporating Lin's framework, Liu et al [8] de-scribed an efficient strategy to count the number of control points required for a complex network, which have an interesting dependence on time constant [9].Structural observability is dual to structural controllability [10].In [11], the requirements of structural observability incorporated explicit use of transitive components of directed graphs -fully connected subgraphs where paths lead from any node to any other node -to identify the minimal number of sites required to observe from a network.
All of these prior works depend critically on the dynamics being linear and generic, in the sense that network connections are essentially random.Joly [12] showed that transitive generic networks with nonlinear nodal dynamics are observable from any node.Nevertheless, symmetries are present in natural networks, as evident from their known structures [13] as well as the presence of synchrony.Recently, Golubitsky et al [14] proved the rigid phase conjecture -that the presence of synchrony in networks implies the presence of symmetries and vice versa.In particular, synchrony is an intrinsic component of brain dynamics in normal and pathological brain dynamics [15].
Our present work is motivated by the question: what role do the symmetries and network coupling strengths play when reconstructing or controlling network dynamics?The intuition here is straightforward: consider 3 linear systems with identical dynamics (diagonal terms of the system matrix A in ẋ(t) = Ax(t)), if the coupling terms are identical (off-diagonal terms of A), it is easy to show that the resulting observability of individual states becomes degenerate as the rows and columns of the system matrix become linearly dependent under elementary matrix operations.For example, consider the trivial case of a 3x3 system matrix of ones: (1 The system is degenerate in the sense that there is only one dynamic, as the rows and columns of A are not independent.This lack of independent rows and columns of the system matrix has direct implications for the controllability and observability of the system.For example, in this trivial system the difference between any two of the states is constrained to a constant x 1 − x 2 = c, thus there is no input coupled to the third state x 3 that could control both x 1 and x 2 independently from each other. In fact, for the more general case of linear time-varying networks, group representation theory [16] has been utilized to show that linear time-varying networks can be non-controllable or non-observable due to the presence of symmetry in the network [17].Brought into context, in networks with symmetry Rubin & Meadows [17] defines a coordinate transform which decomposes the network into decoupled observable (controllable) and unobservable (uncontrollable) subspaces, which then can be determined by inspection like our previous trivial example.Recently, Pecora et al [18] utilized this same method to show how separate subsets of complex networks could sychronize and desychronize according to these same symmetry-defined subspaces.Interestingly, while [17] has been a rather obscure work, it is based on Wigner's work in the 1930's applying group representation theory to the mechanics of atomic spectra [19].Thus, just as the structural symmetry of the Hamiltonian can be used to simplify the solution to the Schrödinger equation [20], the topology of the coupling in a network can have a profound impact on its observation and control.
In this article, we extend the exploration of observability and controllability to network motifs with explicit nonlinearities and symmetries.We further explore the effect of coupling strength within such networks, as well as spatial and temporal effects on observability and controllability.Lastly, we demonstrate the utility of the linear analysis of group representation theory as a tool with which to gain insights into the effects of symmetry in nonlinear networks.

II. BACKGROUND
From the theories of differential embeddings [21] and nonlinear reconstruction [22,23] we can create a nonlinear measure of observability comprised of a measurement function and its higher Lie derivatives employing the differential embedding map [24].The differential embedding map of an observer provides the information contained in a given measurement function and model, which can be quantified by an index [25,26].Computed from the Jacobian of the differential embedding map, the observability index is a matrix condition number which quantifies the perturbation sensitivity (closeness to singularity) of the mapping created by the measurement function used to observe the system.There is a dual theory for controllability, where the differential embedding map is constructed from the control input function and its higher Lie brackets with respect to the nonlinear model function [27,28].Singularities in the map cause information about the system to be lost and observability to decrease.Additionally, the presence of symmetries in the system's differential equations makes observation difficult from variables around which the invariance of the symmetry is manifested [29,30].We extend this analysis to networks of ordinary differential equations and investigate the effects of symmetries on observability and controllability of such networks as a function of connection topology, measurement function, and connection strength.

A. Linear Observability and Controllability
In the early 1960s, Rudolph Kalman introduced the notions of state space decomposition, controllability and observability into the theory of linear systems [5].From this work comes the classic concept of observability for a linear time-invariant (LTI) dynamic system, which defines a 'yes' or 'no' answer whether a state can be reconstructed from a measurement using a rank condition check.
A dynamic model for a linear (time-invariant) system can be represented by where x ∈ R n represents the state variable, u ∈ R m is the external input to the system and y ∈ R p is the output (measurement) function of the state variable.Typically there are less measurements than states, so p < n.The intuition for observability comes from asking whether an initial condition can be determined from a finite period of measuring the system dynamics from one or more sensors.That is, given the system in (2), with x(t) = e At x 0 and Bu = 0, determine the initial condition x 0 from measurement y(t), 0 ≤ t ≤ T .To evaluate this locally, we take the higher derivatives of y(t): Factoring the x terms and putting y and its higher derivatives in matrix form, we have a mapping from outputs to states where the linear observability matrix [31] is defined as The finite limit of taking derivatives in (3) comes from the Cayley-Hamilton theorem, which specifies that any square matrix A satisfies is own characteristic equation, which is the polynomial p(λ) = 0 where p(λ) = (λI n −A).
In other words, A n is spanned by the lower powers of A, from A 0 to A n−1 , Thus, if the observability matrix spans n space (rank(O)= n), the initial condition x 0 can be determined, as the mapping x 0 = O −1 y(t) from output to states exists and is unique.More formally, the system (2) is locally observable (distinguishable at a point x 0 ) if there exists a neighborhood of x 0 such that x 0 = x 1 =⇒ y(x 0 ) = y(x 1 ).In a similar fashion, the linear controllability matrix is derived from asking whether an input u(t) can be found to take any initial condition x(0) = x 0 to arbitrary position x(T ) = x f in a finite period of time T .For the sake of simplicity, we assume a single input u(t) and take the higher derivatives of ẋ(t) = Ax(t) + Bu(t) up to the (n − 1) th derivative of u(t) (again using the Cayley-Hamilton theorem): which gives us a mapping from input to states . . .
where the linear controllability matrix is defined [31] as

B. Differential Embeddings and Nonlinear Observability
From early work on the nonlinear extensions of observability in the 1970s [27,28], it was shown that the observability matrix for nonlinear systems could be expressed using the measurement function and its higher order Lie derivatives with respect to the nonlinear system equations.The core idea is to evaluate a mapping φ from the measurements to the states φ : R p − → R n .In particular, Hermann and Krener [28] showed that the space of the measurement function is embedded in R n when the mapping from measurement to states is everywhere differentiable and injective by the Whitney Embedding Theorem [21,22].An embedding is a map involving differential structure that does not collapse points or tangent directions [23], thus a map φ is an embedding when the determinant of the map Jacobian Det( ∂φ ∂x | ∀x∈R n ) is nonvanishing and one-to-one (injective).In a recent series of papers [24,29,32], Letellier et al. computed the nonlinear observability matrices for the well-known Lorenz and Rössler systems [33,34] and demonstrated that the order of the singularities present in the observability matrix (and thus the amount of intersection between the singularities and the phase space trajectories) was related to the decrease in observability.It is worth noting that the calculation of the observability matrix and locally evaluating the conditioning of the matrix over a state trajectory is a straightforward process and much more tractable than analytically determining the singularities (and thus their order) of the observability matrix of a system of arbitrary order.The former is limited only by computational capacity and the differentiability of the system equations to order n − 1, where n is the order of the system.For a nonlinear system, we replace Ax(t) in (2) by a nonlinear vector field A N L (x(t)), and assume that the smooth scalar measurement function is taken as y(t) = Cx(t) and the system equations comprise the nonlinear vector field f (x(t)) = A N L (x(t)) (note: if there is no external input, then Bu(t) = 0 which we assume here to simplify the display of equations1 ).As in the linear case, we evaluate locally by taking the higher Lie derivatives of y(t), and for compactness of notation dependence on t is implied: where L f (y(x)) is the Lie derivative of y(x) along the vector field f (x).More explicitly, we have x ∈ R n , so as a vector example the first Lie derivative will take the form With formal definitions of the measurement (output) function ( 2) and its higher Lie derivatives (10), the differential embedding map φ is defined as the Lie derivatives L 0 f (y(x)) . . .L n−1 f (y(x)), where the superscripts represent the order of the Lie derivative from 0 to n − 1, where n is the order of the system A N L (x) Taking the Jacobian of the map φ we arrive at the observability matrix . . .
which reduces to (5) for linear system representations.
The key intuition here is that in the nonlinear case the observability matrix becomes a function of the states, where a linear system is always a constant matrix of parameters.

C. Lie Brackets and Nonlinear Controllability
The nonlinear controllability matrix is developed in [27] from intuitive control problem examples and given rigorous treatment in [28]; in a dual fashion to observability, the controllability matrix is a mapping constructed from the input function and its higher order Lie brackets.The Lie bracket is an algebraic operation on two vector fields f (x), g(x) ∈ R n that creates a third vector field F(x), which when taken with g as the input control vector u ∈ R m defines an embedding in R n that maps the input to states [28].
For a nonlinear system, we replace Ax(t) in (2) by a nonlinear vector field A N L (x(t)), take the input function as g = Bu(t) in system (2), and create Lie brackets with respect to the nonlinear vector field f (x(t)) = A N L (x(t)).The Lie bracket is defined as where (ad k f , g) is the adjoint operator and the superscripts represent the order of the Lie bracket.With formal definitions of the input function (2) and its higher Lie brackets (14) from 1 to n, where n is the order of the system matrix A N L (x(t)), the nonlinear controllability matrix is defined as

D. Observability/Controllability Index
In systems with real numbers, calculation of the Kalman rank condition may not yield an accurate measure of the relative closeness to singularity (conditioning) of the observability matrix.It was demonstrated in [25] that the calculation of a matrix condition number [35] would provide a more robust determination of the ill-conditioning inherent in a given observability matrix, since condition number is independent of scaling and is a continuous function of system parameters (and states in the generic nonlinear case).We will use the inverted form of the observability index δ(x) given in [25] where σ min and σ max are the minimum and maximum singular values of O T O respectively and δ(x) = 1 indicates full observability while δ(x) = 0 indicates no observability [36].Similarly the controllability index is just (16) with the substitution of Q for O.

III. OBSERVABILITY AND CONTROLLABILTY OF 3-NODE FITZHUGH-NAGUMO NETWORK MOTIFS A. Fitzhugh-Nagumo System Dynamics
The Fitzhugh-Nagumo (FN) equations [37,38], comprise a general representation of excitable neuronal membrane.The model is a 2-dimensional analogue of the well known Hodgkin-Huxley model [39] of an axonal excitable membrane.The nonlinear FN model can exhibit a variety of dynamical modes which include active transients, limit cycles, relaxation oscillations with multiple time scales, and chaos [37,40].A nonlinear connection function will be used to emulate properties of neuronal synapses.
The system dynamics at a node are given by the (local where i = 1, 2, 3 for the 3-node system, v i represents membrane voltage of node i, w i is recovery, d ij the internodal distance from node j to i, v j the voltage of neighbor nodes with j = 1, 2, 3 and j = i, input current I, and the system parameters a = 0.7, b = 0.8, c = 10.As defined above in Eqns.( 13) and ( 15), the observability and controllability matrices are a function of the states which means a dependence on the particular trajectory taken in phase space.In the following analysis, we are interested in directed information flow between nodes as a function of various topological connection motifs, connection strengths and input forcing functions (which provide different trajectories through phase space).Each motif is representative of a unique combination of directed connections between the 3 nodes with and without latent symmetries.The nonlinear connection function commonly used in neuronal modeling [41] takes the form of the sigmoidal activation function of neighboring activity (a hyperbolic tangent) and an exponential decay with inter-nodal distance.We utilize various coupling strengths to determine the effects on the observability (controllability) of the network.Our coupling function takes the form The sigmoid parameters k = 1, h = 0, m = 1/4, are set such that f N L (v, d) has an output range [0, 1] for the input interval [−2, 2], which is the range of the typical FN voltage variable.To introduce heterogeneity for symmetry breaking a 10% variance noise term was added to each of the d ij terms (there are 6 total possible coupling terms d 12 , d 13 . . .etc.).In this configuration, inputs from neighboring nodes act in an excitatory-only manner, while the driving input current was a square wave (where is the rectangular function, ω = 2π/5 and T = 16 2  3 ) applied to all three nodes to provide a limit cycle regime to the network; for the limit cycle regime generated in the original paper by Fitzhugh [37], the driving current input was constant I = −0.45(with the system parameters mentioned above) which we will also explore.Chaotic dynamics were generated with a slightly different square wave input [40] (with ω = 2π/1.23 and T = 2.7891) also applied to all three nodes.These various driving input regimes allow a wider exploration of the phase space of the system as each driving input commands a different trajectory, which will in turn influence the observability and controllability matrices.

B. Network Motifs and Simulated Data
As we are interested in the effect of connection topology on observability and controllability, we study the simplest nontrivial network: a 3-node network.Such small network motifs are highly overrepresented in neuronal networks [42,43].For each network motif shown in Figure 1, we compute the observability (controllability) indices for various measurement nodes, connection strengths, and driving inputs (dynamic regimes).Measurements of v i for each motif were from each one of the nodes i = 1, 2, or 3. Simulated network data were used to compute the observability (controllability) index for two cases: 1) where the system parameters for all 3 nodes and connections were identical, and 2) where the nodes had a heterogeneous (10% variance) symmetry-breaking set of coupling parameters.To create simulated data, the full six-dimensional FN network equations were in-tegrated from the same initial conditions with the same driving inputs for each node via a Runga-Kutta 4 th order (RK4) method with time step ∆t = 0.04 for 12000 time steps (with the initial transient discarded) in MATLAB for each test case: 1) limit cycle and 2) chaotic dynamical regimes, with a) identical and b) heterogeneous coupling (the nodal parameters remain identical throughout).Convergence of solutions was achieved when ∆t was decreased to 0.004.Data were then imported into Mathematica and inserted into symbolic observability and controllability matrices (computed for each node), which then were numerically computed to obtain the observability (controllability) indices for each coupling strength.The indices were then averaged over the integration paths starting from random initial conditions.These calculations are summarized in Figures 2 -4, 6 and 5 for observability and controllability, in the chaotic, pulsed limit cycle, and constant input limit cycle dynamical regimes.

A. Motifs with Symmetry
For motif 1, the data show that a system with full S 3 symmetry (due to the connection topology and identical nodal and coupling parameters) generates zero observability (controllability) over the entire range of coupling strengths (Figure 2c and 2d).Similarly, no observability (controllability) is seen from node 2 in motif 3 which has a reflection S 2 symmetry across the plane through node 2 (Figure 3c and 3d).Interestingly, the cyclic symmetry of motif 7 does not cause loss of observability (controllability) as shown in Figure 4; motif 7 has rotational C 3 symmetry and valance 1 connectivity (1 input, 1 output).In motifs 1 and 3 the effect of the symmetry is partially broken by introducing a variation in the coupling terms, and the results show non-zero observability (controllability) indices in the plots for such heterogeneous coupling (plots a and b in Figures 2 and 3) with a dependence on the coupling strength.
Of particular interest is the substantial loss of observability (controllability) as the coupling strengths increase to critical levels for systems containing latent structural symmetries in the presence of heterogeneity (motifs 1 and 3, plots a and b in Figures 2 and 3).That is, increasing the coupling strengths when recording (stimulating) from any node in motif 1 or node 2 in motif 3, degrades observability (controllability) as coupling strength increases.A study of the 3D phase plots of the FN voltage variable in motif 1 (as a function of coupling strength for chaotic dynamics) reveals a blowout bifurcation [44] at lower values of coupling strengths (Figure 7), and at higher levels, generalized synchrony [45] and increased observability (controllability), and finally the subsequent decrease in observability (controllability) at the highest levels of coupling strength (motif 1 as observed (controlled) from any node in Figure 2).This is demonstrated in motif 1 (Figure 7), where a bifurcation in the dynamics causes the wandering trajectories at weak coupling strengths to collapse onto the limit cycle attractor at stronger coupling strengths, and at the strongest coupling the dynamics reveal a reverse Hopf bifurcation from limit cycle back into a stable equilibrium.Although motif 7 contains symmetry, the observability and controllability measures appeared unaffected by the presence of this symmetry; further insight into why this happens in such networks requires group representation theory and is presented in section V.

B. Motifs without Symmetry
Local output symmetries occur in motifs 2, and 6 when controlling from the first and second node respectively (green and blue traces in Figure 6), which is remedied by the disambiguating effect of parameter variation.Additionally, as in the motifs with symmetry, the broken local symmetries lose controllability as coupling strength further increases evident in motifs 2 and 6 in Figure 6.In the cases where the indices are zero without symmetries (motifs 5, 6, and 8 in Figures 5 and 6), the motif must contain one or more structurally isolated nodes and hence are not structurally controllable or observable.From the view-point of observability this means that information from the isolated node(s) cannot reach the measured node as the two are not connected in that direction [10,12]; for controllability, this means that the isolated node(s) is not reached by the controlled node due to the two not being connected in that direction [7].This structural nodal isolation is exemplified in motif 8 (in Figures 5 and 6), where the network is only observable from node 1, and only controllable from node 3. Additionally, the plots in Figures 5 and 6 show counterintuitively that as coupling strength increases the observability (controllability) indices can increase to an optimal value, and then begin to decrease as coupling strength increases past this critical coupling value.heterogeneous coupling K, from left to right respectively).In the first row, blue triangles mark locations in phase space where observability is higher than the mean for the trajectory, while the second row contains a phase space trajectory for w and red triangles mark the higher than average controllability.The broken symmetry of the heterogeneous network has trajectories that visit locations in the phase space that vary widely in observability and controllability with a log-normal distribution.

2 3
Motif 1 . Graphic illustration of symmetry axes σn with n = 1, 2, 3 and the cyclic rotation symmetry C3 about an axis perpendicular to the plane of the page.

V. SYMMETRIC NETWORK OBSERVABILITY AND CONTROLLABILITY VIA GROUP REPRESENTATION THEORY
For linear time-varying systems, Rubin & Meadows [17] used the theory of group representations [16,19,20,46] to show how a (circuit) network containing group symmetries would be non-controllable or non-observable due to symmetries (termed NCS or NOS respectively).The analysis involves first determining the irreducible representations of the symmetry group of the system equations, then constructing an orthogonal basis (called a symmetry basis) from the irreducible representations which transforms the system matrix A(t) into block diagonal form (also called modal form).Inspection of the fully transformed system from (2) will reveal if the NCS or NOS property is present via zeros in a critical location of decoupled block-diagonal decomposition ( Â, B, Ĉ) i.e. the form where the transformed system (19) in partitioned form above is non-controllable and non-observable (not completely controllable or observable).This can be seen by inspection, as the zeros present in the partitioned measurement and control functions Ĉ and B leave the transformed system unable to measure or control the mode associated with Z 2 as neither u(t) or Z 1 is present in the equation for Z 2 and Z 2 does not appear in the output.In the next section we summarize the minimum background components of groups and representations (without proofs) in order to further gain insight into how symmetry effects the controllability and observability of our networks.

A. Symmetric Groups and Representations
A symmetry operation on a network is a permutation (in this case nodes) that results in exactly the same configuration as before the transformation was applied.The symmetric group S n consists of all permutations on n symbols -called the order of the group g.The shorthand method of denoting a permutation operation R of nodes in a network will be written (123), where node 1 is replaced by node 2 and node 2 by node 3.This is called a cycle of the permutation [16], and with it we can define all of the permutations of S n .Three of the network motifs studied here contain topological symmetries (Figures 2, 3 and 4); motif 1 has S 3 symmetry, motif 3 has S 2 symmetry and motif 7 contains C 3 symmetry2 , and each of these groups comprise the following sets of permutation operations R R : where E is the identity operation, σ n is a reflection across the n th axis in Figure 8, and C 3 and C 2 3 are two cyclic rotations where C n denotes a rotation of the system by 2π/n radians where the system remains invariant after rotation [20].S 2 and C 3 symmetry in motifs 3 and 7 respectively are subgroups of S 3 The permutation operations R in these symmetric groups can also be represented by monomial matrices3 D(R): where D(R) in ( 22) is a 3-dimensional representation of S 3 group symmetry (for our 3 node motifs); a representation D(R) for S 2 and C 3 group symmetry are just the matrices above in (22) corresponding to the sets of group elements given in (21).
A group of matrices D(•) is said to form a representation of a group S n if a correspondence (denoted ∼) exists between the matrices and the group elements such that products correspond to products, i.e., if [17]); this is known as a homomorphism of the group to be represented, and if the correspondence is one-to-one the representation is isomorphic and called a "faithful" representation of the group.
Theorem 2 from [17] establishes the connection between group theory and the linear network system equations (2), by demonstrating that the monomial representation D(R) of symmetry operations R is conjugate (commutes) with the network system matrix A in (2): where D(R) shows how the states of the system equations transform under the symmetry operation R, and form a reducible representation [16,47] of the symmetric group S n .A representation is said to be reducible if it can be transformed into a block diagonal form via a similarity transformation α, and irreducible if it is already in diagonal form; a reducible representation D(R) that has been reduced to block diagonal form D(R) will have k non-zero submatrices along the diagonal that define the irreducible representations where † represents the complex conjugate transpose of α, l p is the dimension of D (p) (R) and the number of irreducible representations k equals the number of classes the group elements R are partitioned into.This can be found by computing the trace of each representation in D(R), ∀R -called the character of the representationand collecting those that have the same trace into separate classes C p , p = 1 . . .k, which define sets of conjugate elements [20].The character of D(R) is defined as The key to forming irreducible representations in (24) is that the transform α needs to reduce each representation matrix D(R) to diagonal form for every group element R in S n .In (24) the dimension of each irreducible representation l p can be found from the fact that the irreducible representations of the group form an orthogonal basis in the g-dimensional space of the group, and since there can be no more than g independent vectors in the orthogonal basis it can be shown [46] that where the sum is over the number of irreducible representations (or classes of conjugate group elements) k.Some of the irreducible representations D (p) (R) will appear in D(R) more than once while others may not appear at all; the character of the representation completely determines this and the number of times, a p , that D (p) (R) appears in D(R) is defined in [20] as where χ (p) (R) is the trace of D (p) (R), the asterisk denotes complex conjugate and χ(R) is the trace of D(R).
A procedure for transforming the reducible representation D(R) of a symmetry group S n to block diagonal form is presented in [17,47].A unitary transformation α is constructed from the normalized linearly independent columns of the n × n generating matrix where D (p) (R) ii is the (i, i) th diagonal entry of a l pdimensional irreducible representation p (hence i = 1 . . .l p ) of the symmetry group S n and the asterisk denotes complex conjugate.Each matrix G (p) i will contribute a p linearly independent columns from (27) to form the coordinate transformation matrix α.Using equations ( 28) and ( 29) and iterating through all l p rows of each of the k irreducible representations in (24), we 4 For purposes of clarity, we simplified the presentation of the computation of α for our motifs where there is only one set of network nodes that can permuted amongst themselves.For the more general case where the group operations R are separated into subgroups corresponding to different sets of permutable network nodes (e.g.RLC networks, or different neuron types) see [17].
construct α for motif 3 where each linearly independent column of G is a column of α.After normalizing we have which defines the first and second columns of α.Continuing, we have which yields the final column of α (after normalization) Now the coordinate transformation matrix α is Motif 3 in Figure 3 has connection matrix A 3 To control from node 1,2 and 3 respectively, the B matrix takes the form and to observe from node 1,2 and 3 respectively, the C matrix takes the form The block diagonalized system ( Â3 , B, Ĉ) is formed with the substitution Z = α † x, and (A 3 , B, C) in ( 35) to (37) becomes Â3 : By inspection of the transformed system (38) it becomes clear that motif 3 is non-controllable and non-observable from node 2 due to symmetry alone (NCS and NOS), i.e. the transformed system in modal coordinates is NCS and NOS as the mode associated with Z 3 cannot be reached by the input B2 nor can its measurement be inferred from the output Ĉ2 as in (19).The procedure to reduce motif 1 is accomplished in similar fashion 5 and the connection matrix A 1 and its reduced form Â1 is: while the transformed B and C matrices in (36) and (37) are: At first glance it appears that motif 1 is NCS and NOS for measurement and control from node 1 only, and fully controllable and observable from node 2 and 3, however there is a subtle nuance to the controllability and observability of the diagonal form used in [17] and consolidated in (19) to show non-controllability and non-observability by inspection.
It is well known that every non-singular n x n matrix has n eigenvalues λ n and n linearly independent eigenvectors, and that a matrix with repeated eigenvalues of algebraic multiplicity m i will have a degeneracy 1 ≤ q i ≤ m i associated with the number of linearly independent eigenvectors for repeated eigenvalue λ i .This degeneracy q i is also called the geometric multiplicity of λ i , and is equal to the dimension of the null space of A − Iλ i [48].When utilizing similarity transforms to reduce a matrix to diagonal (modal) form this degeneracy in the eigenvectors (brought about by repeated eigenvalues) results in a transformed matrix that is almost diagonal, called the Jordan form matrix.The Jordan form is comprised of submatrices of dimension m i -called Jordan blocks -that have ones on the super-diagonal of each Jordan block J i associated with the generalized eigenvectors of a repeated eigenvalue λ i .The diagonal form in ( 19) is a special case of Jordan form where the matrices on the diagonal are Jordan blocks of dimension one.This is known as the fully degenerate case with q i = m i , and the Jordan form will have m i separate 1 x 1 Jordan blocks associated with each eigenvalue λ i .
The observability and controllability of systems in Jordan form hinges on where the zeros appear in the partitioned C i and B i matrices, where subscript i indicates a partition associated with a particular Jordan block J i .Given in [48,49] the conditions for controllability and observability of a system in Jordan form are: 1.The first columns of C i or the last rows of B i must form a linearly independent set of vectors {c 11 . . .c 1qi } or {b 1e . . .b qie } (subscript e indicates the last row) corresponding to the q i Jordan blocks J λi 1 . . .J λi qi for repeated eigenvalue λ i 2. c 1p = 0 or b pe = 0 when there is only one Jordan block J λi p associated with eigenvalue λ i 3.For single output and single input systems, the partitions of C i and B i are scalars -which are never linearly independent -thus each repeated eigenvalue must only have one Jordan block J λi i associated with it for observability or controllability respectively.
From these criteria, we can now see that the transformed system for motif 1 in (40) contains three 1 x 1 Jordan blocks, two of which are associated with the repeated eigenvalue λ 2 = −1, which violates condition 3); thus we conclude it is NCS and NOS.

C. Motif 7 and Networks Containing Only Rotation Groups
In [17], it was shown how the rth component of α vanishes according to the matrices D (p) (R r r ), where R r r represents a subgroup of the group operations (R) that transform the rth state variable into itself.Subsequently, two theorems were proven that make use of this fact to simplify the analysis of networks that have a single input or output coupled only to the rth state variable, which is precisely parallel to our analysis in section IV.A paraphrasing of Theorem 6 and 12 from [17] for controllability and observability states that such a single input or output network is NCS or NOS if and only if there is an for some value of i, where s r r is +1 or −1 as R r r transforms state variable x r into itself with a plus or minus sign 6 .For this theorem to hold, the equality in (42) must be checked for all possible p for D (p) (R) that appear in D(R) via (27).
Applying (42) to motif 7, the irreducible representations for C 3 symmetry are: where ω = e 2πi 3 .From the subset ( 21) of ( 22) we find that the only operation R r r that leaves either node 1, 2 or 3 ( state variables x 1 , x 2 , or x 3 ) invariant is just the identity operation E, and it is straightforward to see that (42) = 0 for all choices of p, i and r since there is only one group operation that leaves the rth state variable invariant, R r r = E, for r = 1, 2, 3. Thus, motif 7 cannot be NCS or NOS and must be controllable and observable from any node.Corollary 1 to Theorem 6 from [17] contains and expands this result directly to any network with only rotational symmetry (i.e.C n groups), with the caveat that a network with a state variable that is invariant under all the group operations (motif 7 doesn't have such a state variable) will be NCS and NOS if the input and output are coupled to that variable.
These representational group theoretic results explain our nonlinear results in section IV, and clearly demonstrate that different types of symmetry have different effects on the controllability and observability of the networks containing them.While we explicitly assume system matrices with zeros on the diagonal (for simplicity of the calculations) these results hold with generic entries on the diagonal as long as those entries are chosen to preserve the symmetry (e.g. the system matrix A for motif 1 and 7 has a 11 = a 22 = a 33 and motif 3 has a 11 = a 33 , not shown).Linearization of the system equations in (17) would result in a system matrix A with a non-zero diagonal [9], and is typically done in the analysis of nonlinear networks [18] when utilizing such linear analysis techniques.Our computational results demonstrate the utility of this approach in providing insight into the controllability and observability of complex nonlinear networks that have not been linearized.

D. Application to Structurally Controllability (Observability)
It is interesting to note that the demonstration of our results above and those in [17] complement and expand Lin's seminal theorems on structural controllability [7].Essentially, a network with system matrix A and input function B (the pair (A, B)) are assumed to have two types of entries, non-zero generic entries, and fixed entries which are zero.The position of the zero entries leads to the notion of the structure of the system, where different systems with zeros in the same locations are considered structurally equivalent.With this definition of structure, we arrive at the definition for structurally controllability which states that a pair (A , B ) is structurally controllable if and only if there exists a controllable pair (A , B ) with the same structure as (A , B ).The major assumption of this work is that a system deemed to be structurally controllable could indeed be uncontrollable due to the specific entries in A and B, which for a practical application are assumed to be uncertain estimates of the system parameters and thus subject to modification.While Lin's theorems did not explicitly cover symmetry, any network pair (A, B containing symmetry implies constraints on the non-zero entries in (A, B), which is necessary to guarantee that symmetry is present.Thus considering only [7], a network with symmetry could be structurally controllable (observable [10]) as long as the graph of the system contains no dilations 7 or isolated nodes, but NCS (NOS) due to the symmetry.These two theorems together paint a more complete picture of controllability (observability) than either alone as shown in section IV and V, where both are used in concert to explain and understand why certain network motifs were not controllable or observable from particular nodes.Structural controllability (observability) is a more general result, as it does not depend on the explicit non-zero entries of the system pair (A, B) (necessary, but not sufficient), while a network that has the NCS (NOS) property is due to specific sets of the non-zero entries in (A, B) that define the symmetry contained by the system.
Additionally, [7] defined two structures called a "stem" (our motif 8 controlled from node 3) and a"bud" (our motif 7 controlled from any node) which are always structurally controllable.While both are easily shown to be structurally controllable [7], including Theorem 6 and its Corollary 1 from [17] we can take this a step further and declare that any "bud" network (of arbitrary size) containing only rotations is not only structurally controllable, but also fully controllable (or never NCS).The dual of these structures for observability is also defined in [10], and Theorem 12 and its Corollary 1 from [17] completes the statement in a similar fashion for observability.Since networks containing only rotation groups or 7 Defined in the appendix.
"buds" in Lin's terminology are always controllable, we see that in some cases, symmetries alone will not destroy the controllability of structurally controllable networks.

VI. DISCUSSION
Despite the growing importance of exploring observability and controllability in complex graph directed networks, there has been little exploration of nonlinear networks with explicit symmetries.We here report, to our knowledge, the first exploration of symmetries in nonlinear networks, and show that observability and controllability are a function of the specific type of symmetry, the spatial location of nodes sampled or controlled, the strength of the coupling, and the time evolution of the system.
In networks with structural symmetries, group representation theory provides deep insights into how the specific set of symmetry operations possessed by a network will influence its observability and controllability, and can aid in controller or observer design by obtaining a modal decomposition of the network equations into decoupled controllable and uncontrollable (observable and unobservable) subspaces.This knowledge will permit the intelligent placement of the minimum number of sensors and actuators that render a system containing symmetry fully controllable and observable.Additionally, breaking symmetry through randomly altering the coupling strengths established substantial observability or controllability that was absent in the fully symmetric case.In cases where increasing the overall level of coupling strength decreased the observability (controllability), such strong coupling eventually pushed the system towards or through a reverse Hopf bifurcation from limit cycle to a stable equilibrium point, where the lack of dynamic movement of the system then severely decreased the observability (controllability).Intuitively this results from the Lie derivatives (brackets) becoming small as the rate of change of the system trajectories goes to zero.The sensitivity of observability and controllability to the trajectories taken through phase space implies that the choice of control input to a system has to be selected carefully as a poor choice could drive the system into a region that has little to no controllability or observability, thereby thwarting further control effort and/or causing observation of the full system to be lost or limited.Furthermore, when using an observer model for observation or control the regions of local high observability could be utilized to optimize the coupling of the model to a real system by only estimating the full system state when the system transverses observable regions of phase space.
Observation (control) in motifs 2, 3, 4, 5 and 6 suggests a relationship between the degree of connections into and out of a node and its effective observability (controllability).In general, the more direct connections into an observed node, the higher the observability from that node, and the duality suggests that the more direct num-ber of outgoing connections from a controlled node leads to higher controllability than from other less connected nodes.The high degree 'hub' nodes were not the most effective driver nodes in complex networks using linear theory [8], and extending nonlinear results to more complex networks with symmetries is a challenge for future work, which may benefit from linear analysis of the connection topology utilizing group representation theory.
When observing kinematics and dynamics of rigid body mechanics obeying Newton's laws with SE(3) group symmetry, such symmetries must be preserved in constructing an observer (controller) [50].In the observation of graph directed networks containing transitive networks, one can observe from any point equivalently within such transitive components [11].In the control of graph directed networks, the minimum number of control points were related to the maximal matching nodes [8].In [51], contraction theory was used to determine symmetric synchronous subspaces -these spaces actually correspond to our regions without observability or controllability.In fact, the proof of observability is that initial conditions and trajectories do not contract [12].Furthermore, it is clear that the groupoid input equivalence classes (such as our motifs 6 and 7, see figure 21 in [52]) are not equivalently observable or controllable -note that only 1 node can serve as an observer node in motif 6 regardless of coupling strength (our Figure 5).Indeed, whether virtual networks [51] with particular groupoid equivalent symmetries serve as detectors of observability and controllability remains unresolved at this time.
Our deep knowledge of symmetries and observers in classical mechanics [50] do not readily translate to graph directed networks.Further development of a theory of observability and controllability for nonlinear networks with symmetries is a vital open problem for future work.bian of (A2).In this FN network the observability matrix is dependent on the state variables and is thus a function of the location in phase space as the system evolves in time.Letellier et al. [32] used averages of the observability index over the state trajectories in phase space as a qualitative measure of observability.We adopt this convention when computing observability of various network motifs.The indices are computed for each time point in the trajectory, and then the average is taken over all of the trajectories.
Constructing the nonlinear controllability matrix for motif 1 from node 1 begins with the control input function g = Bu(t) = [1, 0, 0, 0, 0, 0] T and its Lie bracket with respect to the nonlinear vector field f in (A1).We exclude the internal driving square wave function here since it is connected to all three nodes, would provide no contribution in the Lie bracket mapping, and we are interested in the mapping from the control input g to the states in order to determine if the system can be controlled, where ∂g ∂x = 0 since g is the same at each node, ∂fi ∂xj is the partial derivative of the i th row of the nonlinear vector field f (x) with respect to the j th state variable, and g i is the i th component of the input vector g.We construct the controllability matrix from the definitions in equations (14 and 15), as the control input function g and its higher Lie Brackets from (ad 1 f , g) to (ad 5 f , g) with respect to the nonlinear vector field system equations, Q = g, [f , g], [f , [f , g]], (ad 3 f , g), (ad 4 f , g), (ad 5 f , g) (A4)

Observability and Controllability Index Distribution
Log-scaled histograms (Figure 9) of the index distributions reveal that the local observability (controllability) along the trajectories in phase space are close to a lognormal distribution.After removing zeros from the data, these log-normal distribution fits were computed and verified with the χ 2 test metric for all of the observability and controllability computation cases that contained an adequate number of data points to accurately compute the fit (over 90% of the data).The χ 2 test for goodness of fit confirmed that the data come from a log-normal distribution with 95% confidence.This type of zeros-censored log-normal distribution is known as a delta distribution [53], and the estimated mean κ and variance ρ 2 are adjusted to account for the proportion of data points that are zero, δ, as follows δ = #{i : where µ and σ are the mean and variance associated with the lognormal distribution computed from the non-zero data.We use these equations to compute the statistics in the plots in the results section (Figures 2 to 7).

1 FIG. 2 .
FIG.2.Calculation of observability (a and c) and controllability (b and d) indices for motif 1 for a chaotic dynamical regime, as measured from each node (green =1, blue × =2, red • =3).The thick lines and symbols mark the mean values of each distribution of indices for each coupling strength, while the smaller symbols and dotted lines represent the ±1 standard deviation confidence intervals.Plots in the top row represent the results computed with symmetry breaking heterogeneous couplings while plots in the bottom row are those with identical coupling strengths.

FIG. 3 . 7 FIG. 4 .
FIG.3.Same as Figure2, except calculations are for motif 3. The calculations show that the reflection symmetry in the network topology causes zero observability and controllability for the symmetric case of observing or controlling from node 2 with identical coupling strengths (c and d).

Fitzhugh- 1 FIG. 6 .FIG. 7 .
FIG.5.Calculation of observability indices for each of the FN network motifs with no underlying group symmetries for a pulsed input limit cycle dynamical regime, as measured from each node (green =1, blue × =2, red • =3).The thick lines and symbols mark the mean values while the smaller symbols and dotted lines represent the ±1 standard deviation confidence intervals.Plots in the top row are computed with heterogeneous couplings while identical coupling strengths are in the bottom row.The calculations show the effect of network coupling strength on observability; motifs 5, 6 and 8 show no observability from node 3 in motif 5, and nodes 2 and 3 in motifs 6 and 8 due to structural isolation.
FIG. 9.The histogram of the log-scaled controllability indices for motif 1 with heterogeneous coupling and chaotic dynamics 5σ

ρ 2 =
(1 − δ)e 2µ+σ 2 (e σ 2 − (1 − δ)), TableIand from(27) appear 1, 0 and 2 times in D(R) respectively.Forming the generating matrix in equation (29) we construct α for motif 1 as follows G independent row of G is a column of α, and thus column of α.We know from (27) that D(2) (R) appears zero times in D(R) and thus yields no contribution to α. Continuing, we have the last two computations from the 2-dimensional irreducible representation D(3) (one for each row)

TABLE I .
Irreducible representations for S3 symmetry.