OnsagerNet: Learning stable and interpretable dynamics using a generalized Onsager principle

We propose a systematic method for learning stable and physically interpretable dynamical models using sampled trajectory data from physical processes based on a generalized Onsager principle. The learned dynamics are autonomous ordinary differential equations parametrized by neural networks that retain clear physical structure information, such as free energy, diffusion, conservative motion, and external forces. For high-dimensional problems with a low-dimensional slow manifold, an autoencoder with metric-preserving regularization is introduced to find the low-dimensional generalized coordinates on which we learn the generalized Onsager dynamics. Our method exhibits clear advantages over existing methods on benchmark problems for learning ordinary differential equations. We further apply this method to study Rayleigh-Bénard convection and learn Lorenz-like low-dimensional autonomous reduced-order models that capture both qualitative and quantitative properties of the underlying dynamics. This forms a general approach to building reduced-order models for forced-dissipative systems.

Figure 1

Computational graph of OnsagerNet for (a) affine $f\left(h\right)$ and (b) nonlinear $f\left(h\right)$.

Figure 2

Accuracy of learned dynamics by using three different ODE neural networks and five different activation functions for the nonlinear damped pendulum problem with $\kappa =4$ and $\gamma =3v^2$. (a) Testing MSE accuracy. The height of the bars stands for$-{log}_{10}$ of the testing MSE. The results are averages from training with three different random seeds. The heights of the red crosses on top of the bars indicate standard deviations. (b) Relative error of predictions versus time for three ODE neural networks with the ReQUr activation function.

Figure 3

Learned energy functions and the exact energy functions in three problems: (a) ReQUr OnsagerNet results for the Hookean model with $k=4$ and $\gamma =3$, (b) ReQU OnsagerNet results for the pendulum model with $k=4$ and $\gamma ={3\left|v\right|}^2$, and (c) ReQU OnsagerNet results for the double-well potential $U\left(x\right)={(x^2-0.5)}^2$ with $\gamma =3$.

Figure 4

Results of the learned ODE model by ReQU OnsagerNet for the nonlinear damped pendulum problem. In the right plot, the dots are trajectory snapshots generated from exact dynamics and the solid curves are trajectories generated from learned dynamics. The red cross is the fixed point numerically calculated for the learned ODE system. Note that the time period of sampled trajectories used to train the OnsagerNet is $T=5$.

Figure 5

Results of the learned ODE model by ReQUr OnsagerNet for the Lorenz system (19, 20, 21) for $b=\frac{8}{3}, \sigma =10$, and $r=16$. In the bottom plots, the dots are trajectory snapshots generated from exact dynamics, the solid curves are trajectories generated from learned dynamics, and the thick curve with red dots is the one with the largest numerical error. The small red crosses are the fixed points numerically calculated for the learned ODE system. The yellow closed curves are the unstable limit cycles calculated from the learned ODE system.

Figure 6

Results of the learned ReQU OnsagerNet model for the Lorenz system (19, 20, 21) for $b=\frac{8}{3}, \sigma =10$, and $r=28$. In the bottom plots, the dots are trajectory snapshots generated from exact dynamics and the solid curves are trajectories generated from learned dynamics. The thick curve with red dots is the one with the largest numerical error. The two red crosses are the (unstable) fixed points numerically calculated for the learned ODE system.

Figure 7

Accuracy of learned dynamics by using OnsagerNets and MLP-ODEN with $ReQU$ and $ReQUr$ as activation functions for learning the Lorenz system with $r=16,28$. The height of the bars stands for $-{log}_{10}$ of the testing MSE plus 3.5; the higher the better. The heights of the red crosses on top of the bars indicate standard deviations. The results are averages of trainings using three different random seeds.

Figure 8

Typical training MSE curves to learn the Lorenz system (19, 20, 21) ($r=28$) with overparametrized neural ODE networks: (a) MLP-ODEN with ReQUr, (b) MLP-ODEN with tanh, (c) OnsagerNet with ReQUr, and (d) OnsagerNet with tanh.

Figure 9

(a) Relative variance of the first 32 principal components and (b) the sample trajectories for the Rayleigh-Bénard convection problem with $r=28$ projected on the first three components. The trajectories are generated using Lorenz-type initial values with random amplitude.

Figure 10

Training and testing loss of OnsagerNet and MLP-ODEN for learning the RBC problem with $r=28$: (a) OnsagerNet $+$ PCA with $m=7$ and (b) MLP-ODEN $+$ PCA with $m=7$.

Figure 11

First three principal components of one trajectory (trajectory 4) in the training set and one (trajectory 89) in the testing set for the RBC problem with $r=28$ and the corresponding simulation results of learned reduced models by OnsagerNets with $m=3$ and 7, respectively: (a) trajectory 4 with $m=3$, (b) trajectory 4 with $m=7$, (c) trajectory 89 with $m=3$, and (d) trajectory 89 with $m=7$.

Figure 12

Six representative trajectories from the sample data (red dots) and the corresponding ODE solutions (solid blue curves) evolved from the same initial values using the learned ODE system for the RBC problem with $r=28$ and $m=7$ for (a) projection to the $(h_1,h_2)$ plane and (b) projection to the $(h_1,h_3)$ plane. The four stable critical points (red crosses) of the learned ODE system are numerically calculated and plotted.

Figure 13

Learned free energy by OnsagerNet with PCA and an autoencoder for the RBC problem with $r=28$. The dependence of the energy function on the first three principal components are shown: (a) PCA with $m=3$, (b) PCA with $m=7$, and (c) autoencoder $+$ OnsagerNet with $m=3$.

Figure 14

Four representative trajectories from the sample data (red dots) and the corresponding ODE solutions (solid blue curves) evolved from the initial values using the learned ODE system ($m=11$) for the case $r=84$ for (a) projection to the $(h_1,h_2)$ plane and (b) projection to the $(h_1/2+h_3,h_2)$ plane. The four unstable critical points (red crosses) and four stable limit cycles (black curves) of the learned ODE system are also plotted.

Figure 15

First three principal components of an exact trajectory and the corresponding simulation results of learned reduced models by OnsagerNets trained using 11 principal components for the RBC problem with $r=84$.

Figure 16

Some representative trajectories from the sample data (colored dots) and the corresponding solutions of learned 11-dimensional OnsagerNet ODEs (solid blue curves) from the same initial values for the RBC problem with (a) $r=28$ and (b) $r=84$. The red crosses are fixed points calculated from the learned ODE systems.

Figure 17

Long time comparison of trajectories originating from an initial condition converging to $A$ for (a) $r=28$ and (b) $r=84$. Black dots are RBC data and colored curves are predictions by the learned OnsagerNet.

Figure 18

(a) Learned potential (with $m=11$) and (b) underlying potential by OnsagerNet for the RBC problem. The isosurfaces in the first three principal component dimensions are shown.

Figure 19

Eigenvalues of the diffusion matrix $M\left(h\right)$ representing local dissipation rates along a line from $A$ to $B$ in the $r=28$ and $m=9$ case. We can see a clear deviation from the linear model where $M$ is constant.