Sparse model selection via integral terms

Model selection and parameter estimation are important for the effective integration of experimental data, scientific theory, and precise simulations. In this work, we develop a learning approach for the selection and identification of a dynamical system directly from noisy data. The learning is performed by extracting a small subset of important features from an overdetermined set of possible features using a nonconvex sparse regression model. The sparse regression model is constructed to fit the noisy data to the trajectory of the dynamical system while using the smallest number of active terms. Computational experiments detail the model's stability, robustness to noise, and recovery accuracy. Examples include nonlinear equations, population dynamics, chaotic systems, and fast-slow systems.

Figure 1

Logistic Growth. The left plot displays the log of the relative ${\ell }^2$ error on the learned coefficients versus the parameter $\delta$. The data are simulated with $t\in [0,10], \mathrm{\Delta }t=0.01$, and $x_0=0.01$. The parameters are fixed at $\epsilon =1.1\sqrt{n}\sigma$ and ${\gamma }^2=0.5$. The noise level is fixed at $2\%$ with the same realizations between experiments. The jump at between $\delta =0.3$ and $\delta =0.4$ marks a transition, where the approximation identifies the correct terms (and continues to refine the values). On the right, the data are plotted with $t\in [0,10], \mathrm{\Delta }t=0.01, x_0=0.01, \epsilon =1.1\sqrt{n}\sigma$, and ${\gamma }^2=0.5$. The noise level is $5\%$ and the error between the learned solution and the original data in the $L^2$ norm is $9.4\times {10}^{-3}$.

Figure 2

Logistic Growth. The data are simulated over $t\in [0,10]$, with time-step $\mathrm{\Delta }t=0.01$, and initial data $x_0=0.01$. The learning parameters are set to $\epsilon =1.1\sqrt{n}\sigma$ and ${\gamma }^2=0.5$. The noisy data are plotted with red markers (darker gray), the interval of strongest influence is marked in green (lighter gray), and the learned solution is in black. On the left ($2\%$ noise level), the interval of strongest influence is given by $[1.5,5]$ and on the right, (noise level $5\%$) the interval of strongest influence is given by $[1.25,6.5]$.

Figure 3

U.S. Census Data. Using the U.S. census data (population in millions) collected from 1790 to 1990, with time-step $\mathrm{\Delta }t=10$, we fit the data without assuming the model form. The learning parameters are set to $\epsilon =1$, and ${\gamma }^2=1.9\times {10}^{-3}$. The learned governing equation corresponds to a parameterized logistic equation and agrees closely with data.

Figure 4

FitzHugh-Nagumo Model. The data use $t\in [0,50], \mathrm{\Delta }t=0.01$, and $x_0={(1,2)}^T$. The parameters are set $\mathrm{to}\epsilon =1.1\sqrt{n}\sigma , {\gamma }^2=0.05$, and noise level $9\%$. The left graph plots the approximate solution in black and in the right graph the solution is resimulated using the learned governing equation in black.

Figure 5

Fast-Slow Approximation. The data use $t\in [0,50], \mathrm{\Delta }t=0.01, x_0={(5,2,1,1)}^T$, and ${\epsilon }_1=5\times {10}^{-3}$. The learning parameters are set to $\epsilon =0.1$ and ${\gamma }^2=0.1$. The left figure plots the noisy data projected onto the $xy$ plane with noise level $3.5\%$. The right figure plots the noisy data along with the data-driven approximation.

Figure 6

Pendulum Problem. The data use $t\in [0,18], \mathrm{\Delta }t=0.1, x_0={(1,1)}^T$ (left) and $x_0={(0.01,0.1)}^T$ (right), $\epsilon =1.1\sqrt{n}\sigma$, and ${\gamma }^2=0.2$. The noise level is fixed at $2.5\%$. On the left, the data-driven approximation (black) uses a polynomial and trigonometric feature space. On the right, the data-driven approximation (black) uses a polynomial feature space. The plot on the right is in the small-angle regime and the correct linearized system is identified.

Figure 7

Lorenz System. The data use $t\in [0,10], \mathrm{\Delta }t=0.001$, and $x_0={(-5,1,20)}^T$. The parameter is $\epsilon =1.1\sqrt{n}\sigma$. The left plot is the noisy data projected onto the $xy$ plane with noise level $3\%$. The right plot is the noisy data along with the data-driven approximation.

Figure 8

Comparison, Logistic Growth. The data are simulated over $t\in [0,15]$, with time-step $\mathrm{\Delta }t=0.01$, and initial data $x_0=0.01$. The black curves are the various solutions of the ordinary differential equation (ODE) when using the learned coefficients for each model. In both plots, two terms are chosen from the 15 term trial set. The first plot shows the result of a sparse optimization method for fitting the trial functions directly to the ODE, inspired by the trial functions used in [3, 4, 5, 6] but using a similar model to Eq. (7). The terms 1 and $x^{14}$ were chosen by the method. The second plot shows the solution using the method in this work, which selected the terms $x$ and $x^2$.