Metalearning Generalizable Dynamics from Trajectories

We present the interpretable meta neural ordinary differential equation (iMODE) method to rapidly learn generalizable (i.e., not parameter-specific) dynamics from trajectories of multiple dynamical systems that vary in their physical parameters. The iMODE method learns metaknowledge, the functional variations of the force field of dynamical system instances without knowing the physical parameters, by adopting a bilevel optimization framework: an outer level capturing the common force field form among studied dynamical system instances and an inner level adapting to individual system instances. A priori physical knowledge can be conveniently embedded in the neural network architecture as inductive bias, such as conservative force field and Euclidean symmetry. With the learned metaknowledge, iMODE can model an unseen system within seconds, and inversely reveal knowledge on the physical parameters of a system, or as a neural gauge to “measure” the physical parameters of an unseen system with observed trajectories. iMODE can be generally applied to a dynamical system of an arbitrary type or number of physical parameters and is validated on bistable, double pendulum, Van der Pol, Slinky, and reaction-diffusion systems.

Figure 1

(a) In iMODE, a neural module ${\mathbf{F}}_{\mathit{\theta }}$ parameterized by $\mathit{\theta }$ takes the concatenation of system state $\mathbf{y}$ and the adaptation parameters $\mathit{\eta }$ and generates the estimated force as output. (b) The bilevel iteration process in the iMODE method. The NN weights $\mathit{\theta }$ are shared across system instances while $\mathit{\eta }$ is adapted for each instance. The metagradient with respect to $\mathit{\theta }$ aggregates the gradients evaluated with instance-adapted $\mathit{\eta }$. (c) Examples of estimated force field ${\mathbf{f}}_{\mathit{\theta }}(·;\mathit{\eta })$ for Van der Pol system instances that differ in their $\epsilon$ parameter (in ascending order from top to bottom). The estimation quality is further evaluated through the trajectories generated by the fields as shown in (d). (d) The estimated force field can be used to predict system trajectories for unseen initial conditions through integration [Eq. (2)]. The signature limit cycles of Van der Pol systems are faithfully reproduced.

Figure 2

(a) The metalearning results for the pendulum. The iMODE trajectory prediction (circles) with different arm lengths (different colors) match those of the ground truth (solid lines). (b) The learned $\mathit{\eta }$ is in good correlation with the effective stiffness of different pendulums ($1/L$). (c) The predicted trajectories (circles) match those of ground truth (solid lines) with different initial conditions (black stars) and different system parameters (different colors) for the bistable system. (d) Two principal axes can be identified from the latent space of the learned $\mathit{\eta }$, each regarding the variation of one physical parameter. (e) Similar to (c) but for the Van der Pol system. (f) The principal axis with respect to the variation of $\omega$ for the Van der Pol system.

Figure 3

(a) Comparison of iMODE test adaptation vs training from scratch on (50) unseen bistable system instances with randomly chosen physical parameters. iMODE demonstrates fast adaptation and good generalization within the first five adaptation steps. (b) The true and learned potential energy functions for the wall bouncing system. The width of the potential well increases as the adaptation parameter increases. (c) The number of top PCA components that preserve a significant portion ($>99\%$) of the variance gives a good estimation of the dimension of true physical parameters. (d) The diffeomorphism constructed by NODE for the bistable system. It shows how a grid in the physical space is continuously deformed into the latent space of adaptation parameters. (e) The mean error and computation time of the neural gauge for 100 systems with randomly generated unseen parameters.

Figure 4

(a) The testing performance of the iMODE model on an unseen Slinky (of an unseen Young’s modulus) with the same boundary condition as the training dataset. The top row is ground truth and the bottom row is the iMODE model prediction at 0.28, 0.47, 0.65, 0.83 s (left to right). The mean squared error of the 3D Slinky reconstruction over the entire trajectory is $8.0\times {10}^{-4}{\mathrm{m}}^2$. (b) The testing performance of the iMODE model on unseen initial and boundary conditions. The top row is ground truth and the bottom row is the iMODE model prediction at 0.15, 0.32, 0.48, 0.65 s (left to right). The mean squared error of the 3D Slinky reconstruction over the entire trajectory is $12.6\times {10}^{-4}{\mathrm{m}}^2$.