Evolutional deep neural network

The notion of an evolutional deep neural network (EDNN) is introduced for the solution of partial differential equations (PDE). The parameters of the network are trained to represent the initial state of the system only and are subsequently updated dynamically, without any further training, to provide an accurate prediction of the evolution of the PDE system. In this framework, the network parameters are treated as functions with respect to the appropriate coordinate and are numerically updated using the governing equations. By marching the neural network weights in the parameter space, EDNN can predict state-space trajectories that are indefinitely long, which is difficult for other neural network approaches. Boundary conditions of the PDEs are treated as hard constraints, are embedded into the neural network, and are therefore exactly satisfied throughout the entire solution trajectory. Several applications including the heat equation, the advection equation, the Burgers equation, the Kuramoto Sivashinsky equation, and the Navier-Stokes equations are solved to demonstrate the versatility and accuracy of EDNN. The application of EDNN to the incompressible Navier-Stokes equations embeds the divergence-free constraint into the network design so that the projection of the momentum equation to solenoidal space is implicitly achieved. The numerical results verify the accuracy of EDNN solutions relative to analytical and benchmark numerical solutions, both for the transient dynamics and statistics of the system.

Figure 1

Schematic representation of PINN and EDNN. (a) PINNs are trained to minimize a cost function comprised of equation residual and data over space and time. (b) The evolution of EDNN, where the network is updated with a direction $\gamma$ calculated from the PDE. The update of neural network parameters represents the evolution of the solution.

Figure 2

Physical domains of PINN and EDNN. (a) PINN is trained and represents the solution on the whole spatiotemporal domain. (b) EDNN only represents the solution on space at one time instant. The time evolution of a single network produces the solution trajectory. The network can be evolved indefinitely.

Figure 3

Schematics for Dirichlet boundary conditions. (a) Geometric quantities including ${\mathit{x}}_e, {\mathit{x}}_w, {\mathit{x}}_n, {\mathit{x}}_s$ and $a_e, a_e, a_e, a_e$ corresponding to point $\mathit{x}$. (b) Geometrical quantities are used to construct a network that satisfies Dirichlet boundary condition.

Figure 4

Schematics for imposing divergence-free constraint. The shaded regions show the auxiliary network $\mathit{v}$ and $\widehat{\mathit{u}}=\mathcal{G}\left(\mathit{v}\right)$ which satisfies the divergence-free constraint.

Figure 5

Numerical solution and error evaluation for 2D heat equation using EDNN. (a), (b) Contours of true and EDNN solution (case 2h) at $t=0.2$. (c) Comparison between true and EDNN solutions (case 1h) at different times and $y=1.0, ⚫ ⚫ ⚫$: true solution, $——$ EDNN solution.

Figure 6

Error evaluations of numerical solution for 2D heat equation using EDNN. (a) Error of EDNN solution versus time, normalized by $L^2$ norm of initial condition. (b) Error normalized by $L^2$ norm of instantaneous true solution $u\left(t\right)$. $······$: case 1h, $–––$: case 2h, $——$: case 2h with lower initial error.

Figure 7

Numerical solution of linear wave equation using EDNN. (a) Spatial solution from case 2lw every 0.2 time units. Symbols: $⚫ ⚫ ⚫$: true solution, $——$ EDNN solution. (b) Relative error: $······$: case 1lw, $–––$: case 2lw.

Figure 8

Numerical solution of N-wave formation using EDNN. (a) Solution at $t=\{0.0,0.1,0.2\}$. (b) Solution at $t=0.32$. Symbols: $⚫ ⚫ ⚫$: true solution, $——$ EDNN solution.

Figure 9

Numerical solution of one-dimensional Kuramoto Sivashinsky equation using EDNN. (a) Numerical solution from spectral discretization; (b) case 2k; (c) case 3k.

Figure 10

Temporal evolution of errors in KS solution using EDNN relative to Fourier spectral method. $······$: case 1k; $–––$: case 2k; $——$: case 3k. Errors $\epsilon$ are reported in (a) linear and (b) logarithmic scale.

Figure 11

Analytical and EDNN solution of Taylor-Green vortex at $t=0.2$. Color contours show the vorticity, and lines are the streamfunction. (a) Analytical solution. (b) Case 6t using EDNN.

Figure 12

Quantitative assessment of EDNN solution for Taylor Green vortex. (a) Decay of kinetic energy from EDNN and analytical solutions. (b) Relative error in EDNN prediction versus the time-step $\mathrm{\Delta }t$. $▲$: cases 1t to 4t; $⚫$: cases 5t to 8t; $■$: case 9t.

Figure 13

Comparison of instantaneous vorticity $\omega$ in Kolmogorov flow using EDNN and spectral method. Colored contours are from case 1kfE (EDNN) and line contours are from 1kfS (spectral). (a) $t=0.25$, (b) $t=0.50$, (c) $t=0.75$, and (d) $t=1.0$.

Figure 14

Instantaneous (a) horizontal and (b) vertical velocities in the turbulent state at $t={10}^5$ withe forcing wave number $n=4$, simulated using EDNN.

Figure 15

Instantaneous snapshots and long-time statistics of the chaotic Kolmogorov flow with forcing wave number $n=3$. (a) Horizontal and (b) vertical velocities in the turbulent state at $t={10}^5$ simulated using EDNN. (c), (d) Statistics of horizontal and vertical velocities, respectively, evaluated from spectral simulation (solid line, case 2kfs) and EDNN (dashed lines, case 2kfE). Black lines are the mean velocities, and blue lines are the root-mean-squared fluctuations.