Optimal reconstruction of dynamical systems: A noise amplification approach

In this work we propose an objective function to guide the search for a state space reconstruction of a dynamical system from a time series of measurements. These statistics can be evaluated on any reconstructed attractor, thereby allowing a direct comparison among different approaches: (uniform or nonuniform) delay vectors, PCA, Legendre coordinates, etc. It can also be used to select the most appropriate parameters of a reconstruction strategy. In the case of delay coordinates this translates into finding the optimal delay time and embedding dimension from the absolute minimum of the advocated cost function. Its definition is based on theoretical arguments on noise amplification, the complexity of the reconstructed attractor, and a direct measure of local stretch which constitutes an irrelevance measure. The proposed method is demonstrated on synthetic and experimental time series.

Figure 1

Profiles of the proposed cost function as a function of the dimension $m$ and time window $t_w$ for the Mackey-Glass time series. The thick dashed line corresponds to the case of using all delay coordinates within the time window $t_w$ (i.e., using $\tau =1$ and $m=t_w+1$, with $\tau$ and $t_w$ expressed in units of sampling time). The cost function has an absolute minimum at $m=4$, $t_w=30$.

Figure 2

Color-coded local cost function in two-dimensional projections of $m$-dimensional delay coordinate reconstructions of the Mackey-Glass attractor. The time window $t_w=30$ is the same for all reconstructions but the dimension increases from left to right. (a) $m=2$; (b) $m=3$; (c) $m=4$.

Figure 3

Comparison of prediction errors (normalized root mean squared error) of local linear models for reconstructions obtained following the standard approach ($\mathrm{MI}+\mathrm{FNN}$, dashed line) versus the proposed methodology (solid line) for Mackey-Glass [panels (a) and (d)], Rössler [panels (b) and (e)] and Lorenz [panels (c) and (f)]. (Top panels) Forecasting error at a fixed horizon as a function of the database fraction ($k/N$) employed for model building. (Bottom panels) Prediction error at a range of horizons $T$ from short to long for a fixed number of neighbors. The horizon $T$ is measured in units of the first recurrence time.

Figure 4

Schematic representation of the measurement process and reconstruction of the attractor of a dynamical system. This figure is inspired in Casdagli et al.[4]. See the main text for details.

Figure 5

Schematic representation of two DC reconstructions, (a) ${\Phi }_1$ and (b) ${\Phi }_2$, with ${\tau }_1<{\tau }_2$. The distance between neighbors increases with the delay time $\tau$ due to the stretching and folding of the attractor, while the global characteristic size $L$ of the attractor remains constant because it only depends on the amplitude of the time series.

Figure 6

Response of the characteristic radii ${\epsilon }_k$ ($k=2$) to changes in the reconstruction parameters $t_w$ and $m$ for the Lorenz time series (noise free and with 10% noise). (a) ${\epsilon }_k$ monotonically grows with $t_w$ and $m$ in the noise-free case. (b) The same holds true for the noisy case. (c) For the normalized radii ${\epsilon }_k^{*}={\epsilon }_k/\sqrt{m}$ the dependence with $m$ is eliminated at the lower end of $t_w$ values. (d) In the noisy case—as well as for large $t_w$ in the noise-free case—the sampled points fill the $m$-dimensional space and the growth of ${\epsilon }_k$ with $m$ cannot be avoided. In all cases the thick dashed line corresponds to the full delay coordinate (fDC) reconstruction ($\tau =1$ and $m=t_w+1$).

Figure 7

(Color online). Profiles of $L_k$ vs $t_w$ for a range of $k$ values and benchmark case studies. (a) Lorenz, $m=3$; (b) Rössler, $m=3$; (c) Mackey-Glass, $m=4$; (d) Mackey-Glass with 10% noise, $m=4$. For all time series studied in this work the structure of maxima and minima converges for low values of $k$.

Figure 8

Schematic representation of two reconstructions of the same region of an attractor with (a) well-behaved orbits and (b) collapsed ones. The computation of $E_k\left(T\right)$ using $k=2$ will yield identical results for both reconstructions while $L_k$ will penalize the collapsed orbits of panel (b).

Figure 9

Illustration of the sensitivity of $L_k$ and $E_k$ to an orbit collapse for the three-dimensional DC reconstruction of the Rössler system. (a) Plot of $L_k$ and $E_k$ vs $\tau$. Notice that at $\tau =7$ the curves differ maximally. (b) Detail of the reconstructed attractor at $\tau =7$. The reconstruction exhibits collapsed orbits at this specific delay value.

Figure 10

Sensitivity of $L_k$ to collapsed orbits. The local cost function ${\alpha }_k^2{\sigma }_k^2(T,\overline{x})$ is plotted vs $x_0$ in the top panels and color coded in the two-dimensional projection of the reconstruction in the bottom panels for reconstruction parameters $m=3$ and (a) $\tau =2$, (b) $\tau =5$, (c) $\tau =7$, and (d) $\tau =11$.

Figure 11

Same as Fig. 10 for $E_k^2(T,\overline{x})$. This quantity is less sensitive to a collapse of orbits [panel (c)] and false neighbors [panel (d)] as compared to its ground level, which presents a spurious structure [panels (a) and (b)]. See the main text for details.

Figure 12

(Color online). Optimal window size $t_w$ (in an $L_k$ sense) as a function of the horizon parameter $T_M$ for (a) the noise-free Mackey-Glass case and (b) the noisy case. Panels (a) and (b) also show $t_w$ vs $T$ as obtained by applying Gao’s method (dashed gray line) which exhibits a high dependence on parameter $T$. The dashed vertical lines signal the horizon parameter $T_M$ equal to the first maximum of the upper curve of the space-time separation plot[54] for the time series without any reconstruction [panel (c)].

Figure 13

(Color online). Cost function vs time window size $t_w$ for the Mackey-Glass series plus 10% noise. (a) Delay coordinate reconstructions with different dimensions $m$ as in Fig. 1. (b) Legendre coordinates reconstruction of different dimensions $n$. Both panels show the curve for the fDC reconstruction ($\tau =1$ and $m=t_w+1$). Note that the Legendre profiles reach values below the latter curve. The vertical dashed line indicates $t_w={\tau }_w^{*}$.

Figure 14

Chua’s circuit data[56]. (a) The full-length time series. (b) Detail of the first 1500 data points. The characteristic period is approximately 94 sampling time units. (c) Preliminary two-dimensional DC reconstruction using $\tau =23$ (which corresponds to a quarter of the characteristic period).

Figure 15

Cost function vs time window size $t_w$ for Chua’s circuit time series. Panel (a) corresponds to DC reconstructions with different dimensions $m$ as in Fig. 1. Panel (b) corresponds to Legendre coordinates of different dimensions $n$. Both panels show the profile for the fDC reconstruction ($\tau =1$ and $m=t_w+1$). The vertical dotted line indicates the value of ${\tau }_w^{*}$.

Figure 16

Views of the three-dimensional Legendre coordinate reconstruction of Chua’s circuit attractor. (a) $y_1$ vs $y_0$; (b) $y_2$ vs $y_0$; and (c) $y_2$ vs $y_1$. (d) Perspective view of the reconstructed attractor. The local cost function over the attractor is color coded. The principal directions of the attractor (in the PCA sense) are aligned with the coordinate axes. The local cost function does not present localized regions of high values [as, e.g., in Fig. 2b], implying the absence of false neighbors.

Figure 17

SFI Laser data. (a) Full-length time series. (b) Detail of the first 1000 samples. (c) Zoom on the first 100 points. The characteristic period is approximately seven sampling time units.

Figure 18

Evaluation of the cost function for different reconstruction strategies applied to the SFI laser time series. (a) $L_k$ vs window size $t_w$ for $m$-dimensional DC reconstructions and for the fDC reconstruction (thick dashed gray line). (b) As a function of the dimension $m$, we plot (i) $L_k$ for fDC (thick dashed gray line), (ii) min${}_{t_w}{L}_k$ for DC, (iii) min${}_{t_w}{L}_k$ for Legendre coordinate reconstructions, and (iv) $L_k$ for PCA reconstructions. In the latter case, principal components are incorporated in decreasing order of their corresponding eigenvalues (solid line).

Figure 19

Views of a three-dimensional reconstruction of the SFI laser attractor using PCA coordinates. (a) $y_1$ vs $y_0$; (b) $y_6$ vs $y_0$; and (c) $y_6$ vs $y_1$. (d) View in perspective of the reconstructed attractor. The local cost function for the seven-dimensional PCA reconstruction is color coded over the attractor and penalizes regions of high divergence. A spline interpolation was used to connect points in order to guide the eye for this undersampled time series.