Revealing the state space of turbulence using machine learning

Despite the apparent complexity of turbulent flow, identifying a simpler description of the underlying dynamical system remains a fundamental challenge. Capturing how the turbulent flow meanders among unstable states (simple invariant solutions) in phase space, as envisaged by Hopf [Commun. Pure Appl. Math. 1, 303 (1948)], using some efficient representation offers the best hope of doing this, despite the inherent difficulty in identifying these states. Here, we make a significant step toward this goal by demonstrating that deep convolutional autoencoders can identify low-dimensional representations of two-dimensional turbulence which are closely associated with the simple invariant solutions characterizing the turbulent attractor. To establish this, we develop latent Fourier analysis that decomposes the flow embedding into a set of orthogonal latent Fourier modes which decode into physically meaningful patterns resembling simple invariant solutions. The utility of this approach is highlighted by analyzing turbulent Kolmogorov flow (monochromatically forced flow on a 2D torus) at $\text{Re}=40,$ where, in between intermittent bursts, the flow resides in the neighborhood of an unstable state and is very low dimensional. Projections onto individual latent Fourier wavenumbers reveal the simple invariant solutions organizing both the quiescent and bursting dynamics in a systematic way inaccessible to previous approaches.

Figure 1

Autoencoders for dimensionality reduction in Kolmogorov flow. (a) Loss $\frac{1}{\left|\text{data}\right|}{\sum }_{\text{data}}{\parallel [\mathcal{D}\circ \mathcal{E}]\left(\omega \right)-\omega \parallel }_2^2$ as a function of embedding dimension $m$. (b) Dissipation PDFs of autoencoded vorticity fields. The background orange is obtained from the original test data set, the overlayed blue PDFs are for $m\in \{96,48,32,3\}$ (also labeled in the figure). (c) Decodes of a set of snapshots of increasing dissipation for various embedding dimensions $m$. Snapshots are ordered left to right by dissipation, running from $D/D_l\approx 0.08$ to $D/D_l\approx 0.26$, indicating, for example, that $m=3$ captures low-dissipation episodes well but struggles to represent high-dissipation events.

Figure 2

An overview of the approach to performing latent Fourier decompositions on (latent) representations of physical systems with a continuous symmetry. A latent Fourier decomposition is performed by constructing an operator to map between embeddings of shifted versions of the same snapshot. Note that only four latent wavenumbers, $\{0,1,2,3\}$, are required for the monochromatically forced turbulence considered here. In the project and decode step (bottom right), the projection within the $l=0$ subspace is also included (see the discussion in the text). For comparison, the projection onto $k=0$ is also included in the projection onto the physical Fourier modes.

Figure 3

Eigenvalue spectra of the latent symmetry operators for various embedding sizes $m$ and shifts $\alpha$. Panels (a)–(c) have fixed $\alpha =2\pi /9$ and varying $m$: (a) $m=16$, (b) $m=32$, (c) $m=64$. Panels (d) and (e) have fixed $m=96$ and varying $\alpha$: (d) $\alpha =2\pi /9$, (e) $\alpha =2\pi /12$. Finally, panel (f) displays the eigenvalues of a numerical approximation to the vertical shift-reflect operator, $\mathbf{S}$ at $m=96$. In all cases, the outer black circle is $\left|\mathrm{\Lambda }\right|=1$, the grey interior circle is $\left|\mathrm{\Lambda }\right|=0.9$. The rays from the origin identify the relevant roots of unity.

Figure 4

Eigenvalue spectra of a latent symmetry operator and decodes of the projection onto individual eigenspaces for example snapshots. (a) Seven example snapshots of vorticity, $\omega$. (b) Eigenvalue spectrum of ${\mathbf{T}}_{\alpha }$ on the $m=96$ autoencoder with $\alpha =2\pi /9$. Note that only half the spectrum is shown. (c)–(f) Decodes of the snapshots in (a) after their embeddings are projected onto individual latent Fourier eigenspaces [latent wavenumber $l$ is identified in each row and by the red line from the eigenvalue spectrum (b)]. Note that the $l=0$ subspace is always included for decodes of latent wavenumbers $l>0$.

Figure 5

Visualization of the degenerate eigenspace $l=1$ via decodes of projections onto its PCA modes for an example snapshot. (a) Singular values from the $l=1$ subspace. (b) An example vortical snapshot (to the left of the dashed black line) and the decode of the projection of its embedding onto individual PCA modes $\left\{{\mathit{u}}_n^{\left(1\right)}\right\}$ within the $l=1$ subspace; note the $l=0$ projection is also included [see Eq. (6)]. (c) As (b) but only the first PCA mode from the $l=0$ subspace is used [see Eq. (7)].

Figure 6

The primary bifurcation of Kolmogorov flow and its connection to the primary latent Fourier mode within the $l=1$ subspace. (a) Equilibrium states at $\text{Re}\approx \{14,20,40,60\}$ from the solution branch which bifurcates from the basic laminar state at $\text{Re}\approx 9.97$. (b) Decode of the projection of the embeddings of the equilibria in (a) onto the first PCA modes within the $l=0$ and $l=1$ subspaces. (c) Amplitude (measured by dissipation) of this equilibrium as a function of Reynolds number (blue line, the laminar solution sits on $1-D/D_l=0$) with the example states shown above identified with symbols [see corresponding symbols in in the panels of (a)]. It should be emphasized that training has been conducted at fixed $\text{Re}=40$, and that all of the training snapshots are from within the turbulent attractor and do not feature this simple equilibrium.

Figure 7

Two-dimensional t-SNE of embeddings of 5000 vorticity snapshots in the $m=96$ network. The figure was created by computing projections of the embeddings onto the five leading PCA modes within each degenerate eigenspace $l\in \{0,1,2,3\}$. Translational dependence of the features was then removed by taking the absolute value of these projections for $l>0$ before the t-SNE algorithm [22] was applied. The data points are colored by their dissipation values which run between $D/D_l=0.06$ (dark blue) and $D/D_l=0.33$ (yellow). Decodes of example snapshots are shown on the right. The eight colored squares represent the embeddings of the same vorticity snapshot (see the decode of the yellow square at bottom right) with the shift-reflect operation repeatedly applied. The orange arrows indicate how this field is moved between sectors of the low-dissipation octagon under applications of a shift-reflect operation; the dashed arrow labeled ${\mathcal{S}}^2$ corresponds to a full wavelength shift in $y$.

Figure 8

Bursting episodes: Their recurrent patterns and associated exact coherent structures. (a), (b) Dissipation and (c), (d) projections onto latent wavenumbers $l=1$ and $l=2$ for a long turbulent signal [panels (b) and (d) show zoomed-in regions identified with gray shading). In (c), (d), the solid blue line is the projection onto the $l=1$ mode corresponding to the primary bifurcation in the flow, ${\gamma }_1:={\parallel {\mathcal{P}}_0^1\left(\mathcal{E}\left(\omega \right)\right)\parallel }^2$ (see text), the dashed blue line is the projection onto the rest of the modes in the $l=1$ eigenspace, ${\gamma }_{1+}:={\parallel {\sum }_{j=1}^{d\left(1\right)-1}{\mathcal{P}}_j^1\left(\mathcal{E}\left(\omega \right)\right)\parallel }^2$; the solid red line is the projection onto the full $l=2$ eigenspace giving ${\gamma }_2$. In panel (d), the dashed red line overlaying the $l=2$ curve identifies regions where $D/D_l>0.15$. Square markers indicate where equilibria and traveling waves corresponding to the $l=2$ recurrent patterns were found; the labels [i]–[iv] next to the green markers coincide with the equilibria reported in panel (e). In panel (e), we show snapshots (left) extracted at the green markers [i]–[iv] in (d), the decode of the projection onto the $l=2$ eigenspace (middle, note this includes the $l=0$ contribution) and the converged equilibria or traveling wave (right).

Figure 9

Some bursting equilibria and traveling waves. (a)–(d) Vorticity field for four equilibria converged from projections and decodes onto $l=2$. The dissipation rate of these solutions is (a) $D/D_l=0.1974$, (b) $D/D_l=0.2042$, (c) $D/D_l=0.2233$, (d) $D/D_l=0.6058$. (e)–(h) Traveling waves. The phase speed and dissipation of these structures is (a) $c=0.0006$, $D/D_l=0.2184$; (b) $c=0.0053$, $D/D_l=0.2257$; (c) $c=0.3178$, $D/D_l=0.4357$; (d) $c=0.1059$, $D/D_l=0.4536$.

Figure 10

Bursting periodic orbit. Snapshots separated by $\delta t=0.5$, labeled on each panel, from a discovered (relative) periodic orbit with period $T=3.41276$ and shift $s=-0.56022$.