Transfer learning for nonlinear dynamics and its application to fluid turbulence

We introduce transfer learning for nonlinear dynamics, which enables efficient predictions of chaotic dynamics by utilizing a small amount of data. For the Lorenz chaos, by optimizing the transfer rate, we accomplish more accurate inference than the conventional method by an order of magnitude. Moreover, a surprisingly small amount of learning is enough to infer the energy dissipation rate of the Navier-Stokes turbulence because we can, thanks to the small-scale universality of turbulence, transfer a large amount of the knowledge learned from turbulence data at lower Reynolds number.

Figure 1

Illustrative example of transfer learning for RC applied to the inference problem of $z\left(t\right)$ from $x\left(t\right)$ of the Lorenz chaos. The red-shaded region on the left shows the data of the Lorenz chaos in the source domain (${\rho }_{\mathcal{S}}=28$). The blue-shaded region on the right shows the data in the target domain (${\rho }_{\mathcal{T}}=40$). (a, ${\mathrm{a}}^{\prime }$) Projection of the attractor onto the $x-z$ plane. (b, ${\mathrm{b}}^{\prime }$) Lorenz plot. The data plotted in (b) is replotted in (${\mathrm{b}}^{\prime }$) for comparison. (c, ${\mathrm{c}}^{\prime }$) Training data in ${\mathcal{D}}_{\mathcal{S}}$ and ${\mathcal{D}}_{\mathcal{T}}$. We consider the case that the training data in ${\mathcal{D}}_{\mathcal{T}}$ is limited.

Figure 2

(a) Generalization error (mean-square error, MSE) of the inference task for the Lorenz equations with ${\rho }_{\mathcal{T}}=32$. The horizontal axis shows the transfer rate $\mu$. The red circles represent the median MSE. The blue shaded area indicates the range of the MSEs from the first to the third quartile. Inset: the logarithmic graph of the same MSE data. The broken and solid lines represent the median MSE in the cases of $\mu =0$ and $\mu =\infty$, respectively. (b) The blue lines are the inferred values ${\widehat{z}}^{\left(j\right)}\left(t\right)(j=1,\cdots ,100)$ by the conventional method ($\mu =0$, upper panel) and the proposed transfer learning ($\mu ={10}^{-8}$, lower panel). The red broken lines represent the answer data $z\left(t\right)$.

Figure 3

Normalized time-series of the energy $K\left(t\right)$ and energy dissipation rate $\epsilon \left(t\right)$ at (a) $R_{\lambda }\simeq 35$ and (b) $R_{\lambda }\simeq 120$. The gray solid line and red broken line represent $K\left(t\right)$ and $\epsilon \left(t\right)$, respectively.

Figure 4

(a) Generalization error (mean-square error, MSE) as a function of the transfer rate $\mu$. Similar plots to Fig. 3 but for the inference of the energy dissipation rate of the Navier-Stokes turbulence at $R_{\lambda }\simeq 120$. The red circles represent the median MSE. The blue shaded area indicates the range of MSEs from the first to the third quartile. Inset: the logarithmic graph of the same MSE data. The broken and solid lines represent the median MSE in the cases of $\mu =0$ and $\mu =\infty$, respectively. (b) The blue lines are the inferred values ${\widehat{\epsilon }}^{\left(j\right)}\left(t\right)(j=1,\cdots ,20)$ at $R_{\lambda }\simeq 120$ by the conventional method ($\mu =0$, upper panel) and the proposed transfer learning ($\mu =1$, lower panel). The red broken lines represent the answer data $\epsilon \left(t\right)$.

Figure 5

MSE (red circles) of the inference task for (a) the Lorenz equations and (b) the Navier-Stokes equations, as a function of the transfer rate $\mu$. The target domain of the Lorenz case is ${\rho }_{\mathcal{T}}=32$. The MSE is estimated with a short test data $T_{\text{test}}=T_L^{\prime }$ and $M=1$. The broken and solid lines represent the MSE in the cases of $\mu =0$ and $\mu =\infty$, respectively.

Figure 6

MSE of the inference task for the Lorenz equations with ${\rho }_{\mathcal{T}}=40$ as a function of the transfer rate $\mu$. The red circles represent the median MSE. The blue shade indicates the range in MSEs from the first to the third quartile. The broken and solid lines represent the median MSE in the cases of $\mu =0$ and $\mu =\infty$, respectively.

Figure 7

The time-series of the inferred values ${\widehat{z}}^{\left(j\right)}\left(t\right)(j=1,...,M)$ at ${\rho }_{\mathcal{T}}=40$ corresponding to the $M=100$ samples of the training data. The inferred values by the conventional ($\mu =0$), the transfer learning ($\mu ={10}^{-8}$), and the simple transfer method ($\mu =\infty$) are shown in (a), (b), and (c), respectively. The red broken lines depict the answer data $z\left(t\right)$.