Ensemble Kalman Filtering without a Model

Methods of data assimilation are established in physical sciences and engineering for the merging of observed data with dynamical models. When the model is nonlinear, methods such as the ensemble Kalman filter have been developed for this purpose. At the other end of the spectrum, when a model is not known, the delay coordinate method introduced by Takens has been used to reconstruct nonlinear dynamics. In this article, we merge these two important lines of research. A model-free filter is introduced based on the filtering equations of Kalman and the data-driven modeling of Takens. This procedure replaces the model with dynamics reconstructed from delay coordinates, while using the Kalman update formulation to reconcile new observations. We find that this combination of approaches results in comparable efficiency to parametric methods in identifying underlying dynamics, and may actually be superior in cases of model error.

Popular Summary

Methods of data assimilation, such as the ensemble Kalman filter used to estimate unknown variables, are becoming increasingly important in physics and geophysics research. Nonparametric methods for prediction have been developed from Takens’s delay-coordinate embedding. Here, we show how these two lines of research can be merged to create a methodology for extracting information from data that is greater than the sum of the parts. We test our new method using a weather data set.

We focus on situations in which variables are poorly constrained and/or models are incomplete (e.g., weather predictions). The goal of our work is to filter out noisy observations and reconstruct the underlying model. Our “Kalman-Takens” filtering method confers the statistical advantages of Kalman filtering without the necessity of applying a physical model, using reconstructed dynamics in place of a model. Since we avoid the use of a model, our results are free of biases due to strong model assumptions. We apply our method to systems with up to 40 dimensions in which observations are corrupted by up to 100% noise. We also consider an El Niño data set collected from January 1950 through December 1999, and we show that our method can filter a noisy time series without relying on a model. Surprisingly, in some cases in which there is large model error, our method is superior to using the model. However, our technique suffers from the limitation that it assumes that noise follows a Gaussian distribution.

We expect that our findings will pave the way for more accurate studies of data sets for which no underlying model is known.

Figure 1

Results of filtering the full 40-dimensional Lorenz-96 system. (a) Spatiotemporal plot of the system dynamics. The horizontal axis represents time, and the vertical axis represents the spatial position. From top to bottom, (1) the true system trajectory, (2) observations perturbed by 100% noise, (3) Kalman-Takens output, and (4) parametric filter output. (b) Spatiotemporal plot of the resulting absolute errors. For readability, only absolute errors greater than 5 are shown. From top to bottom, error in (1) the observed signals, (2) Kalman-Takens output, and (3) parametric filter output.

Figure 2

Given noisy observations of the Lorenz-63 $x$ variable (green circles) sampled at rate $h=0.05$, the goal is to reduce the signal error ($\mathrm{RMSE}=4.76$) using an ensemble Kalman filter where the true $x$ trajectory (solid black curve) is desired. (a) If we have knowledge of the Lorenz-63 equations, the standard filtering methodology can be used whereby the noisy $x$ data are assimilated to the model and using an EnKF a noise-reduced estimate of the $x$ variable as well as estimates of the unobserved $y$ and $z$ variables are provided. We refer to this as the parametric filter (solid blue curve, $\mathrm{RMSE}=2.87$). (b) The Kalman-Takens filter is able to significantly filter the noise in the observed signal (solid red curve, $\mathrm{RMSE}=3.04$) to a level comparable to the parametric filter.

Figure 3

Filter performance of both the parametric (solid blue curve) and nonparametric (solid red curves) filters when denoising the observed Lorenz-63 $x$ variable at increasing levels of (a) large and (b) low observation noise (signal error, dotted black curve). Time series sampled at rate $h=0.05$ and error bars denote standard error over 10 realizations. $Q$ and $R$ noise covariances are estimated adaptively. The nonparametric filter is able to significantly reduce the signal error to a level comparable to the parametric filter which has full use of the model. In some instances, the performance of the nonparametric filter improves by reprocessing the data (dotted red curve). At higher noise levels, the discrepancy between the parametric and nonparametric filter becomes negligible.

Figure 4

Results of forecasting the Lorenz-63 $x$ variable when the training data are perturbed by 30% noise. Results averaged over 3000 realizations. Forecast error calculated with respect to (a) the non-noisy truth and (b) the noisy truth. When the training data are filtered by the Kalman-Takens filter (solid red curve) we gain improved forecasting capability as compared to use of the unfiltered training data (dotted black curve). The forecasting results (using identical forecast algorithms) when the training data are filtered by the parametric filter (solid blue curve) are included for a point of reference.

Figure 5

The filtering problem in a 40-dimensional Lorenz-96 system where noisy observations from 3 of the network nodes are available. Observations are perturbed by 60% noise. The goal is to filter the noisy observations of the $x_1$ node (black circles) and reduce the signal error ($\mathrm{RMSE}=2.16$). Black solid curve denotes true trajectory of variable. (a) The parametric filter (solid blue curve) with full knowledge of the system is able to filter the signal to $\mathrm{RMSE}=1.17$. This requires full estimation of the 40-dimensional state. (b) The nonparametric Kalman-Takens filter (solid red curve), with no knowledge of the underlying physical system and utilizing only delay coordinates of the observables, is able to filter the signal to $\mathrm{RMSE}=1.36$.

Figure 6

Filter performance of the parametric (solid blue curve) and Kalman-Takens filter (solid red curve) when denoising the observed Lorenz-96 $x_1$ variable at increasing levels of observation noise (signal error, dotted black curve). The variable is sampled at rate $h=0.05$ and the error bars denote standard error over 10 realizations. $Q$ and $R$ noise covariances are tuned off-line for optimal performance of the filters. Results presented for both a (a) five-dimensional Lorenz-96 system with only $x_1$ observable and a (b) 40-dimensional Lorenz-96 system where $x_1$, $x_2$, and $x_{40}$ are observable. We assume additional nodes are observable in the 40-dimensional case due to the high dimensionality of the system, which inhibits both the parametric and Kalman-Takens filter. In both systems, the Kalman-Takens filter is able to significantly reduce the signal error to a level comparable to the parametric filter that has full knowledge of the system.

Figure 7

Results of forecasting the $x_1$ node in a five-dimensional Lorenz-96 system when the training data are affected by 60% noise. Results averaged over 2500 realizations. Forecast error calculated with respect to (a) the non-noisy truth and (b) the noisy truth. Even in this higher-dimensional system, we see that use of the Kalman-Takens filter to filter the training data results in improved forecasting capability (solid red curve) as compared to forecasting without filtering (dotted black curve). The forecast based on Kalman-Takens initial conditions compares well to the same Takens-based forecast method using initial conditions from the parametric filter (solid blue curve).

Figure 8

Under situations of model error, the performance of the parametric filter can degrade rapidly. In our examples, this model error is simulated by perturbing the system parameters. Even under this relatively mild form of model error, it is evident that the parametric filter can have difficulty in filtering a noisy observation. Error bars denote standard error over 10 realizations. (a) Results for filtering the Lorenz-63 $x$ variable when the observations are perturbed by 60% noise (dotted black curve indicates signal error). The performance of the parametric filter (solid blue curve) under slight model error becomes much worse than the Kalman-Takens filter (solid red curve), which is completely robust to any form of model error. (b) Results for filtering the $x_1$ node in a 40-dimensional Lorenz-96 system when observations are perturbed by 60% noise. Only three data points are shown for the parametric filter because starting at 30% parameter perturbation the filter diverges, once again showing the crippling effect of model error on the performance of the parametric filter.

Figure 9

The Kalman-Takens method is used to filter a set of historical data from the El Niño 3.4 index. The resulting 14-month lead nonparametric diffusion forecast using the filtered historical data is shown (solid red curve). The forecasted trajectory captures the general pattern of the true trajectory (solid black curve).

Figure 10

Results of the diffusion forecast when trained on the unfiltered historical data (solid black curve) and Kalman-Takens filtered data (solid red curve). (a) Both forecasts compare favorably to the climatological error (dotted gray curve). However, processing the historical data with the Kalman-Takens filter results in improved forecast error. (b) Similarly, forecast correlation is improved by filtering the data.