Predictions of turbulent shear flows using deep neural networks

In the present work, we assess the capabilities of neural networks to predict temporally evolving turbulent flows. In particular, we use the nine-equation shear flow model by Moehlis et al. [New J. Phys. 6, 56 (2004)] to generate training data for two types of neural networks: the multilayer perceptron (MLP) and the long short-term memory (LSTM) networks. We tested a number of neural network architectures by varying the number of layers, number of units per layer, dimension of the input, and weight initialization and activation functions in order to obtain the best configurations for flow prediction. Because of its ability to exploit the sequential nature of the data, the LSTM network outperformed the MLP. The LSTM led to excellent predictions of turbulence statistics (with relative errors of $0.45\%$ and $2.49\%$ in mean and fluctuating quantities, respectively) and of the dynamical behavior of the system (characterized by Poincaré maps and Lyapunov exponents). This is an exploratory study where we consider a low-order representation of near-wall turbulence. Based on the present results, the proposed machine-learning framework may underpin future applications aimed at developing accurate and efficient data-driven subgrid-scale models for large-eddy simulations of more complex wall-bounded turbulent flows, including channels and developing boundary layers.

Figure 1

(Left) Schematic representation of a multilayer perceptron (MLP) with two hidden layers, i.e., $l=2$, where each circle in the diagram depicts a single neuron. The MLP takes as input a $d$-dimensional vector $\chi$, then produces hidden $n$-dimensional vectors ${\zeta }_i$ for each of the two hidden layers, and finally generates an output vector $\zeta$ with $m$ dimensions. Note that the dimension of the input depends on the number of previous temporal values used for the prediction $p$. In this example, $p=500, d=9p=4500, m=9$, and $n=90$, which implies that 90 neurons are used for each of the hidden layers. The transformation at a certain layer $i$ is performed with the model parameters given by the weight matrix ${\mathbf{W}}_i$. The ${\mathbf{W}}_i$ values are determined through learning using a gradient-following optimization technique on training data. (Right) Detail of a neuron, which takes a vector as input and generates a scalar output by performing a vector dot product and a nonlinear operation given by the activation function $g$.

Figure 2

Schematic representation of the training and validation losses as a function of the number of epochs. (Left) Adequate fitting, where the validation loss does not increase with the number of epochs, and (right) overfitting beyond the dashed vertical line (which indicates the point for early stopping).

Figure 3

Turbulence statistics corresponding to (left) streamwise mean profile, (middle) streamwise velocity fluctuations, and (right) Reynolds shear stress. Blue dots are used for the reference nine-equation model, and colored lines denote predictions using the following architectures: (red) LSTM1, (green) LSTM2, and (cyan) LSTM3. The networks were trained with 100 data sets.

Figure 4

Turbulence statistics corresponding to (top left) streamwise mean profile, (top middle) streamwise velocity fluctuations, and (top right) Reynolds shear stress; (bottom left) wall-normal velocity fluctuations, (bottom middle) skewness, and (bottom right) flatness. Blue dots are used for the reference nine-equation model and red lines for the predictions using the LSTM1 network trained with 10 000 data sets.

Figure 5

Instantaneous velocity fields reconstructed from the mode amplitudes for the LSTM1 architecture. The colored contours represent the velocity component perpendicular to the visualized plane, and the vectors are the velocity components in the plane. The left panel is averaged in the streamwise direction, and the right panel shows the flow in the vertical midplane.

Figure 6

(Left) Streamwise, (middle) wall-normal, and (right) spanwise root-mean-square vorticity fluctuations obtained from (blue dots) the reference nine-equation model and (red lines) the LSTM1 network. Note that the network was trained with 10 000 data sets.

Figure 7

Temporal evolution of the nine coefficients for a particular time series, from (blue) the nine-equation model and (red) the LSTM1 network trained with 10 000 data sets.

Figure 8

(Left) Probability density function of the Poincaré maps, where the intersection with the $a_2=0$ plane (with $d{a}_2/dt<0$) is shown. (Right) Ensemble-averaged divergence of instantaneous time series after a perturbation with $|\delta {\mathbf{A}}_0|={10}^{-6}$ is introduced at $t_0=500$, showing initial exponential growth and the value of the Lyapunov exponent (dashed lines added to illustrate the obtained slope). In both panels, blue and red denote reference model and LSTM prediction, respectively.