Nonintrusive reduced order modeling framework for quasigeostrophic turbulence

In this study, we present a nonintrusive reduced order modeling (ROM) framework for large-scale quasistationary systems. The framework proposed herein exploits the time series prediction capability of long short-term memory (LSTM) recurrent neural network architecture such that (1) in the training phase, the LSTM model is trained on the modal coefficients extracted from the high-resolution data snapshots using proper orthogonal decomposition (POD) transform, and (2) in the testing phase, the trained model predicts the modal coefficients for the total time recursively based on the initial time history. Hence, no prior information about the underlying governing equations is required to generate the ROM. To illustrate the predictive performance of the proposed framework, the mean flow fields and time series response of the field values are reconstructed from the predicted modal coefficients by using an inverse POD transform. As a representative benchmark test case, we consider a two-dimensional quasigeostrophic ocean circulation model which, in general, displays an enormous range of fluctuating spatial and temporal scales. We first demonstrate that the conventional Galerkin projection-based reduced order modeling of such systems requires a high number of POD modes to obtain a stable flow physics. In addition, ROM-Galerkin projection (ROM-GP) does not seem to capture the intermittent bursts appearing in the dynamics of the first few most energetic modes. However, the proposed nonintrusive ROM framework based on LSTM (ROM-LSTM) yields a stable solution even for a small number of POD modes. We also observe that the ROM-LSTM model is able to capture quasiperiodic intermittent bursts accurately, and yields a stable and accurate mean flow dynamics using the time history of a few previous time states, denoted as the lookback time window in this paper. We show several features of ROM-LSTM framework such as significantly higher accuracy than ROM-GP, and faster performance using larger time step size. Throughout the paper, we demonstrate our findings in terms of time series evolution of the field values and mean flow patterns, which suggest that the proposed fully nonintrusive ROM framework is robust and capable of predicting chaotic nonlinear fluid flows in an extremely efficient way compared to the conventional projection-based ROM framework.

Figure 1

Schematic representation of a typical LSTM network.

Figure 2

Work-flow diagram of the ROM-LSTM framework. Note that the training phase (offline computation) is computationally heavier compared to the testing (online computation) phase.

Figure 3

Evaluation results for hyperparameters search for LSTM architecture. We perform fivefold cross-validation for different number of layers $\mathcal{L}=4,6,8$ and number of neurons $\mathcal{N}=40,60,80$. The mean of the evaluation metric for five data samples is used to select the hyperparameters. The error bars shows the standard deviation of evaluation metric score for five samples.

Figure 4

Variation of CFL number from time $t=10$ to $t=100$ for the FOM simulation.

Figure 5

Mean stream-function and vorticity fields obtained by the FOM simulation and the standard ROM-GP simulation at Re $=450$ and Ro $=3.6\times {10}^{-3}$ flow condition. (a) ${\psi }_{\text{FOM}}$ at a resolution of $256\times 512$, (b) ${\psi }_{\text{ROM-GP}}$ with $R=10$ modes, (c) ${\psi }_{\text{ROM-GP}}$ with $R=20$ modes, (d) ${\psi }_{\text{ROM-GP}}$ with $R=40$ modes, (e) ${\psi }_{\text{ROM-GP}}$ with $R=80$ modes, (f) ${\omega }_{\text{FOM}}$ at a resolution of $256\times 512$, (g) ${\omega }_{\text{ROM-GP}}$ with $R=10$ modes, (h) ${\omega }_{\text{ROM-GP}}$ with $R=20$ modes, (i) ${\omega }_{\text{ROM-GP}}$ with $R=40$ modes, and (j) ${\omega }_{\text{ROM-GP}}$ with $R=80$ modes.

Figure 6

Time series evolution of the first and tenth modal coefficients, $a_1\left(t\right)$ and $a_{10}\left(t\right),$ respectively, between $t=10$ and $t=100$ for standard ROM-GP simulation at Re $=450$ and Ro $=3.6\times {10}^{-3}$. (a) $a_1\left(t\right)$ for ROM-GP with $R=10$ modes, (b) $a_{10}\left(t\right)$ for ROM-GP with $R=10$ modes, (c) $a_1\left(t\right)$ for ROM-GP with $R=20$ modes, (d) $a_{10}\left(t\right)$ for ROM-GP with $R=20$ modes, (e) $a_1\left(t\right)$ for ROM-GP with $R=30$ modes, (f) $a_{10}\left(t\right)$ for ROM-GP with $R=30$ modes, (g) $a_1\left(t\right)$ for ROM-GP with $R=40$ modes, (h) $a_{10}\left(t\right)$ for ROM-GP with $R=40$ modes, (i) $a_1\left(t\right)$ for ROM-GP with $R=80$ modes, and (j) $a_{10}\left(t\right)$ for ROM-GP with $R=80$ modes. True projection series is underlined in each figure with black straight line. The training zone is shown with orange dashed line (from $t=10$ to $t=50$) and the out-of-sample testing zone is shown with red dashed line (from $t=51$ to $t=100$) in ROM-LSTM solution series in each figure.

Figure 7

Mean stream-function and vorticity fields obtained by the ROM-LSTM simulation based on different lookback time window, $\sigma$ at Re $=450$ and Ro $=3.6\times {10}^{-3}$ flow condition. (a) ${\psi }_{\text{FOM}}$ at a resolution of $256\times 512$, (b) ${\psi }_{\text{ROM-LSTM}}$ with $\sigma =1$, (c) ${\psi }_{\text{ROM-LSTM}}$ with $\sigma =2$, (d) ${\psi }_{\text{ROM-LSTM}}$ with $\sigma =4$, (e) ${\psi }_{\text{ROM-LSTM}}$ with $\sigma =5$, (f) ${\omega }_{\text{FOM}}$ at a resolution of $256\times 512$, (g) ${\omega }_{\text{ROM-LSTM}}$ with $\sigma =1$, (h) ${\omega }_{\text{ROM-LSTM}}$ with $\sigma =2$, (i) ${\omega }_{\text{ROM-LSTM}}$ with $\sigma =4$, and (j) ${\omega }_{\text{ROM-LSTM}}$ with $\sigma =5$. Note that the LSTM model is trained with $R=10$ modes.

Figure 8

Time series evolution of the modal coefficients between $t=10$ and $t=100$ for ROM-LSTM simulation at Re $=450$ and Ro $=3.6\times {10}^{-3}$. Note that the LSTM model is trained with $R=10$ modes and $\sigma =5$. True projection series is underlined in each figure with black straight line. The training zone is shown with orange dashed line (from $t=10$ to $t=50$) and the out-of-sample testing zone is shown with red dashed line (from $t=51$ to $t=100$) in ROM-LSTM solution series in each figure.

Figure 9

PDF for true and predicted modal coefficients between $t=50$ and $t=100$ for ROM-LSTM simulation at Re $=450$ and Ro $=3.6\times {10}^{-3}$. Note that the LSTM model is trained with $R=10$ modes and $\sigma =5$. The training is done using true modal coefficients between $t=10$ and $t=50$.

Figure 10

Mean stream-function and vorticity fields obtained by the ROM-LSTM simulation based on the number of modes to train the LSTM model at Re $=450$ and Ro $=3.6\times {10}^{-3}$ flow condition. (a) ${\psi }_{\text{FOM}}$ at a resolution of $256\times 512$, (b) ${\psi }_{\text{ROM-LSTM}}$ for LSTM training with $R=2$ modes, (c) ${\psi }_{\text{ROM-LSTM}}$ for LSTM training with $R=4$ modes, (d) ${\psi }_{\text{ROM-LSTM}}$ for LSTM training with $R=8$ modes, (e) ${\psi }_{\text{ROM-LSTM}}$ for LSTM training with $R=10$ modes, (f) ${\omega }_{\text{FOM}}$ at a resolution of $256\times 512$, (g) ${\omega }_{\text{ROM-LSTM}}$ for LSTM training with $R=2$ modes, (h) ${\omega }_{\text{ROM-LSTM}}$ for LSTM training with $R=4$ modes, (i) ${\omega }_{\text{ROM-LSTM}}$ for LSTM training with $R=8$ modes, and (j) ${\omega }_{\text{ROM-LSTM}}$ for LSTM training with $R=10$ modes. Note that the LSTM model is trained with $\sigma =5$.

Figure 11

Time series evolution of the modal coefficients between $t=10$ and $t=100$ for ROM-LSTM simulation based on different lookback time windows, $\sigma$ and LSTM training with $R=2$ modes at Re $=450$ and Ro $=3.6\times {10}^{-3}$. (a) $a_1\left(t\right)$ with $\sigma =1$, (b) $a_2\left(t\right)$ with $\sigma =1$, (c) $a_1\left(t\right)$ with $\sigma =2$, (d) $a_2\left(t\right)$ with $\sigma =2$, (e) $a_1\left(t\right)$ with $\sigma =3$, (f) $a_2\left(t\right)$ with $\sigma =3$, (g) $a_1\left(t\right)$ with $\sigma =4$, (h) $a_2\left(t\right)$ with $\sigma =4$, (i) $a_1\left(t\right)$ with $\sigma =5$, and (j) $a_2\left(t\right)$ with $\sigma =5$. True projection series is underlined in each figure with black straight line. The training zone is shown with orange dashed line (from $t=10$ to $t=50$) and the out-of-sample testing zone is shown with red dashed line (from $t=51$ to $t=100$) in ROM-LSTM solution series in each figure.

Figure 12

Mean stream-function and vorticity fields obtained by the ROM-LSTM simulation based on different lookback time windows, $\sigma$ and LSTM training with $R=2$ modes at Re $=450$ and Ro $=3.6\times {10}^{-3}$. (a) ${\psi }_{\text{FOM}}$ at a resolution of $256\times 512$, (b) ${\psi }_{\text{ROM-LSTM}}$ for LSTM training with $\sigma =2$, (c) ${\psi }_{\text{ROM-LSTM}}$ for LSTM training with $\sigma =3$, (d) ${\psi }_{\text{ROM-LSTM}}$ for LSTM training with $\sigma =4$, (e) ${\psi }_{\text{ROM-LSTM}}$ for LSTM training with $\sigma =5$, (f) ${\omega }_{\text{FOM}}$ at a resolution of $256\times 512$, (g) ${\omega }_{\text{ROM-LSTM}}$ for LSTM training with $\sigma =2$, (h) ${\omega }_{\text{ROM-LSTM}}$ for LSTM training with $\sigma =3$, (i) ${\omega }_{\text{ROM-LSTM}}$ for LSTM training with $\sigma =4$, and (j) ${\omega }_{\text{ROM-LSTM}}$ for LSTM training with $\sigma =5$.

Figure 13

Time series evolution of first two modal coefficients, $a_1\left(t\right)$ and $a_2\left(t\right),$ respectively, between $t=10$ and $t=100$ for different ROMs at Re $=450$ and Ro $=3.6\times {10}^{-3}$. (a) $a_1\left(t\right)$ for ROM-GP with $R=10$ modes, (b) $a_2\left(t\right)$ for ROM-GP with $R=10$ modes, (c) $a_1\left(t\right)$ for ROM-LSTM trained with $R=2$ modes, (d) $a_2\left(t\right)$ for ROM-LSTM trained with $R=2$ modes, (e) $a_1\left(t\right)$ for ROM-LSTM trained with $R=4$ modes, (f) $a_2\left(t\right)$ for ROM-LSTM trained with $R=4$ modes, (g) $a_1\left(t\right)$ for ROM-LSTM trained with $R=8$ modes, (h) $a_2\left(t\right)$ for ROM-LSTM trained with $R=8$ modes, (i) $a_1\left(t\right)$ for ROM-LSTM trained with $R=10$ modes, and (j) $a_2\left(t\right)$ for ROM-LSTM trained with $R=10$ modes. True projection series is underlined in each figure with black straight line. The training zone is shown with orange dashed line (from $t=10$ to $t=50$) and the out-of-sample testing zone is shown with red dashed line (from $t=51$ to $t=100$) in ROM-LSTM solution series in each figure.

Figure 14

In-sample and out-of-sample flow parameters used for evaluating interpolatory performance of ROM-LSTM model.

Figure 15

PDF for true and predicted modal coefficients between $t=50$ and $t=100$ for ROM-LSTM simulation at Re $=250$ and Ro $=1.6\times {10}^{-3}$. Note that the LSTM model is trained with $R=10$ modes and $\sigma =5$. The training is done using true modal coefficients between $t=10$ and $t=50$ for (Re, Ro) = (100, $1.6\times {10}^{-3}$), (200, $0.9\times {10}^{-3}$), (200, $1.6\times {10}^{-3}$), (200, $3.6\times {10}^{-3}$), and (450, $3.6\times {10}^{-3}$).

Figure 16

Mean stream-function and vorticity fields obtained by the ROM-LSTM simulation based on the number of modes to train the LSTM model at Re $=250$ and Ro $=1.6\times {10}^{-3}$ flow condition. The model is trained for five different operating conditions and is evaluated for out-of-sample parameters. (a) ${\psi }_{\text{FOM}}$ at a resolution of $256\times 512$, (b) ${\psi }_{\text{ROM-LSTM}}$ for LSTM training with $R=2$ modes, (c) ${\psi }_{\text{ROM-LSTM}}$ for LSTM training with $R=4$ modes, (d) ${\psi }_{\text{ROM-LSTM}}$ for LSTM training with $R=8$ modes, (e) ${\psi }_{\text{ROM-LSTM}}$ for LSTM training with $R=10$ modes, (f) ${\omega }_{\text{FOM}}$ at a resolution of $256\times 512$, (g) ${\omega }_{\text{ROM-LSTM}}$ for LSTM training with $R=2$ modes, (h) ${\omega }_{\text{ROM-LSTM}}$ for LSTM training with $R=4$ modes, (i) ${\omega }_{\text{ROM-LSTM}}$ for LSTM training with $R=8$ modes, and (j) ${\omega }_{\text{ROM-LSTM}}$ for LSTM training with $R=10$ modes. Note that the LSTM models are trained with $\sigma =5$.

Figure 17

Computational overhead for the ROM-GP framework for different number of modes and different time step size $\mathrm{\Delta }t$ for Re = 250 and Ro = $1.6\times {10}^{-3}$ test condition. The number in each box presents the CPU time required for integration of Galerkin projection ODEs from time $t=10$ to $t=100$. NaN means the solution diverges after few time steps.

Figure 18

Mean stream-function and vorticity fields obtained by the ROM-LSTM simulation based on different lookback time windows, $\sigma$ and LSTM training with $R=10$ modes at Re $=250$ and Ro $=1.6\times {10}^{-3}$. The model is trained using 400 snapshots stored at time interval $\mathrm{\Delta }t=1\times {10}^{-1}$ from $t=10$ to $t=50$. (a) ${\psi }_{\text{FOM}}$ at a resolution of $256\times 512$, (b) ${\psi }_{\text{ROM-LSTM}}$ for LSTM training with $\sigma =2$, (c) ${\psi }_{\text{ROM-LSTM}}$ for LSTM training with $\sigma =3$, (d) ${\psi }_{\text{ROM-LSTM}}$ for LSTM training with $\sigma =4$, (e) ${\psi }_{\text{ROM-LSTM}}$ for LSTM training with $\sigma =5$, (f) ${\omega }_{\text{FOM}}$ at a resolution of $256\times 512$, (g) ${\omega }_{\text{ROM-LSTM}}$ for LSTM training with $\sigma =2$, (h) ${\omega }_{\text{ROM-LSTM}}$ for LSTM training with $\sigma =3$, (i) ${\omega }_{\text{ROM-LSTM}}$ for LSTM training with $\sigma =4$, and (j) ${\omega }_{\text{ROM-LSTM}}$ for LSTM training with $\sigma =5$.

Figure 19

Mean stream-function and vorticity fields obtained by the ROM-LSTM simulation based on different lookback time windows, $\sigma$ and LSTM training with $R=10$ modes at Re $=250$ and Ro $=1.6\times {10}^{-3}$. The model is trained using 200 snapshots sampled at time interval $\mathrm{\Delta }t=2\times {10}^{-1}$ from $t=10$ to $t=50$. (a) ${\psi }_{\text{FOM}}$ at a resolution of $256\times 512$, (b) ${\psi }_{\text{ROM-LSTM}}$ for LSTM training with $\sigma =2$, (c) ${\psi }_{\text{ROM-LSTM}}$ for LSTM training with $\sigma =3$, (d) ${\psi }_{\text{ROM-LSTM}}$ for LSTM training with $\sigma =4$, (e) ${\psi }_{\text{ROM-LSTM}}$ for LSTM training with $\sigma =5$, (f) ${\omega }_{\text{FOM}}$ at a resolution of $256\times 512$, (g) ${\omega }_{\text{ROM-LSTM}}$ for LSTM training with $\sigma =2$, (h) ${\omega }_{\text{ROM-LSTM}}$ for LSTM training with $\sigma =3$, (i) ${\omega }_{\text{ROM-LSTM}}$ for LSTM training with $\sigma =4$, and (j) ${\omega }_{\text{ROM-LSTM}}$ for LSTM training with $\sigma =5$.