Forecasting the evolution of nonlinear and nonstationary systems using recurrence-based local Gaussian process models

An approach based on combining nonparametric Gaussian process (GP) modeling with certain local topological considerations is presented for prediction (one-step look ahead) of complex physical systems that exhibit nonlinear and nonstationary dynamics. The key idea here is to partition system trajectories into multiple near-stationary segments by aligning the boundaries of the partitions with those of the piecewise affine projections of the underlying dynamic system, and deriving nonparametric prediction models within each segment. Such an alignment is achieved through the consideration of recurrence and other local topological properties of the underlying system. This approach was applied for state and performance forecasting in Lorenz system under different levels of induced noise and nonstationarity, synthetic heart-rate signals, and a real-world time-series from an industrial operation known to exhibit highly nonlinear and nonstationary dynamics. The results show that local Gaussian process can significantly outperform not just classical system identification, neural network and nonparametric models, but also the sequential Bayesian Monte Carlo methods in terms of prediction accuracy and computational speed.

Figure 1

(Color online) (a) Lorenz time-series and (b) state portrait obtained from delay embedding of the time-series, showing a reconstructed Lorenz attractor embedded in a three-dimensional delay coordinate space [30].

Figure 2

(Color online) A 2D projection of a reconstructed Lorenz attractor with a schematic illustration of partitioning of the attractor into three affine segments: ${\mathbf{U}}_{\mathbf{1}}$ and ${\mathbf{U}}_{\mathbf{3}}$ contain trajectories about foci ${\overline{x}}^1$ and ${\overline{x}}^3$, and ${\mathbf{U}}_{\mathbf{2}}$ about a saddle point ${\overline{x}}^2$.

Figure 3

(Color online) (a) Unthresholded recurrence plot of a Lorenz attractor with black vertical lines marking the true boundaries between two piecewise affine segments; (b) variation of correlation ${\rho }_k$ between the adjacent columns $k$ and $k+1$ of the recurrence matrix, with the green (light gray) vertical lines marking the boundaries as in (a), indicating that the boundaries between two piecewise affine segments are marked by local minima of the correlation ${\rho }_k$; (c) segmentation of the recurrence plot using correlation change rate $\Delta {\rho }_k$, with the green (light gray) vertical lines representing the boundaries between different segments. The boundaries are obtained by applying the segmentation threshold $\Delta {\rho }^{\ast }$ determined from the distribution of $\Delta {\rho }_k$ values. The results indicate that the local maxima of $\Delta {\rho }_k$, that are deemed statistical outliers, serve as effective boundaries between piecewise affine segments. Note that the method also tends to further partition a piecewise stationary segment into multiple such segments. However, this does not have a significant effect on prediction accuracy.

Figure 4

(a) A 1000 data points long segment of first component of Lorenz-like system time-series, and (b) 2D projection of its corresponding phase portait. Here, the sampling interval is 0.01 time units.

Figure 5

(Color online) Segmentation of Lorenz-like system attractor with a representative point $x_{\ast }∊{\mathbf{U}}_{\mathbf{1}}$ (shown as a gray annular ring) whose evolution needs to be predicted using samples taken from the interior of the segment ${\mathbf{U}}_{\mathbf{1}}$ (shown as green stars), at the boundary of ${\mathbf{U}}_{\mathbf{1}}$ (shown as red squares), and/or samples from ${\mathbf{U}}_{\mathbf{2}}$ that are just across the boundary from ${\mathbf{U}}_{\mathbf{1}}$ (shown as red circles).

Figure 6

(a) A histogram showing the distribution of SNR values of the noise added to various segments, (b) Lorenz-like nonstationary system time-series, and (c) a 2D projection of the corresponding phase space. Here the sampling interval is 0.01 time units.

Figure 7

(Color online) A highly nonlinear and nonstationary synthetic physiological heart-rate signal (top) and the piecewise stationary segment boundaries marked by green (light gray) vertical lines obtained from applying the correlation change rate criterion (bottom). Here the sampling frequency is set to the average heart rate of the measured time-series.

Figure 8

(Color online) A representative segment of the synthetic physiological heart rate signal (red line) with the corresponding one-step look-ahead prediction from LGP (blue dashed line) and a 95% confidence limit (blue shade) indicating that the LGP model is able to predict the evolution of the heart rate time-series $\left(R^2=71\mathrm{\%}\right)$, and that 93% of the actual realizations were within the 95% of LGP-predicted confidence limits.

Figure 9

(Color online) (a) Automobile assembly line throughput time-series; (b) One-step prediction with 95% confidence interval for MGP; (c) One-step prediction with 95% confidence interval for LGP, showing that 84% of the actual realizations lie within the 95% confidence interval compared to 51% for MGP. Here, each time unit indicates a production shift (an 8 h period).

Figure 10

Comparison of accuracies for prediction of throughputs of assembly line machines using alternative methods indicating that among all the models tested, MGP and LGP sustain the least RMSE and highest prediction $R^2$ values.