Boosting reservoir computer performance with multiple delays

Time delays play a significant role in dynamical systems, as they affect their transient behavior and the dimensionality of their attractors. The number, values, and spacing of these time delays influences the eigenvalues of a nonlinear delay-differential system at its fixed point. Here we explore a multidelay system as the core computational element of a reservoir computer making predictions on its input in the usual regime close to fixed point instability. Variations in the number and separation of time delays are first examined to determine the effect of such parameters of the delay distribution on the effectiveness of time-delay reservoirs for nonlinear time series prediction. We demonstrate computationally that an optoelectronic device with multiple different delays can improve the mapping of scalar input into higher-dimensional dynamics, and thus its memory and prediction capabilities for input time series generated by low- and high-dimensional dynamical systems. In particular, this enhances the suitability of such reservoir computers for predicting input data with temporal correlations. Additionally, we highlight the pronounced harmful resonance condition for reservoir computing when using an electro-optic oscillator model with multiple delays. We illustrate that the resonance point may shift depending on the task at hand, such as cross prediction or multistep ahead prediction, in both single delay and multiple delay cases.

Figure 1

Schematic of the single delay RC. The time-multiplexed masked input is injected into the nonlinear node, and travels along the loop with a clock cycle of $T$, here made equal to the delay $\tau$. The virtual nodes provide time series that are combined in the output layer of the reservoir computer. These nodes are separated by a time interval of $\theta =\frac{T}{N}$. The time arguments for the virtual nodes illustrate how they span the state of the nonlinear delay system across the time interval $(K-1)T$ to $KT$, where $K$ is a positive integer. Only the first and last virtual nodes are shown.

Figure 2

Linear stability analysis of Eq. (1) for a multidelay RC for a system with (a) two delays and (b) five delays. As the number of delays increases, the peak of the stability region shifts towards smaller values of delay spacing $\mathrm{\Delta }\tau$ and larger feedback strengths.

Figure 3

Visualizing time-delay couplings between virtual nodes in the loop depicted in Fig. 1, showcasing various combinations of single and two delays with different time delay values and spacing between delays. Each vertical block (i.e., cycle of duration $T$) corresponds to a distinct input data point, which is a sample of the driving (input) time series. Within each block, there are $N$ virtual nodes with a spacing of $\theta =\frac{T}{N}$. A single time delay is assumed in the first column. In (a), the time delay is set to the clock cycle ($\tau =T$). The dynamics can then be seen as mapping (i.e., affecting the dynamics of) each virtual node in one block to itself in the next block. In (c), the time delay is $\tau =T-\theta$, resulting in inner couplings where each virtual node maps backwards by $\theta$. In this case, the dynamics of a given node in one block affects the dynamics of the previous node in the next block. Furthermore, the first node of each block also affects the dynamics of the last node in the same block. The second column depicts cases where each node's dynamics are affected by two delayed states. Here solid lines are associated with the smaller delay, and dashed lines with the larger delay. In (b), ${\tau }_1=T$ and ${\tau }_2=2T$ ($\mathrm{\Delta }\tau =T$), corresponding to the resonance case for single-step ahead prediction in multidelay reservoir computing; this case is more susceptible to poor performance. In (d), ${\tau }_1=T-\theta$ and ${\tau }_2=2T$, so $\mathrm{\Delta }\tau ={\tau }_2-{\tau }_1=T+\theta$. In (f), ${\tau }_1=T+\theta$ and ${\tau }_2=2T$, so $\mathrm{\Delta }\tau =T-\theta$.

Figure 4

Results of Lorenz $x$ task prediction errors for three distinct cases of (a) $\mathrm{\Delta }\tau$=38, (b) $\mathrm{\Delta }\tau$=39, and (c) $\mathrm{\Delta }\tau$=39.8, all with $M=5$ when ${\tau }_{\text{min}}=39.8$ and $T=40$. Each panel displays the input scaling ($\gamma$) on the $x$ axis against the feedback strength ($\beta$) on the $y$ axis.

Figure 5

Prediction error plots over a range of delay intervals ($\mathrm{\Delta }\tau$) and minimum delays (${\tau }_{\text{min}}$) centered on the values used in Fig. 4 for Lorenz $x$ task after fixing $\beta$ and $\gamma$ to their respective optimal values found for each of the equivalent panels in Fig. 4 (that resulted in the lowest error for each case).

Figure 6

Impact of minimum time delay (${\tau }_{\text{min}}$) on the Lorenz $x$ time series prediction, illustrated for four different samples, each obtained from unique initial conditions. The black line depicts the average of the samples' errors.

Figure 7

MDRC can improve prediction and cross-prediction compared to single delay RC. (a) One-step ahead time series prediction error of the Lorenz $x$ component as a function of ${\tau }_{\text{min}}$ for 1, 2, and 5 time delay RC. (b) Same as in (a) but for one-step ahead cross-prediction of the Lorenz $z$ component with the input being the Lorenz $x$ component. The vertical red dashed lines indicate the value of the cycle time $T=40$. Parameters are in Table 1 and $\theta =0.2$.

Figure 8

One-step ahead time series prediction of the Lorenz $z$ component when the input is the Lorenz $z$ component for different ${\tau }_{\text{min}}$. The parameters used are in Table 1.

Figure 9

The normalized root mean square (NRMSE) for a 34-step ahead prediction of the Mackey-Glass time series. The spacing between delays, denoted as $\mathrm{\Delta }\tau$, is given in the inset.

Figure 10

A comparison of prediction errors in systems with five delays (black) and ten delays (blue) is depicted for the Mackey-Glass task.

Figure 11

Prediction error of a RC system with five delays, as a function of the parameters ${\tau }_{\text{min}}$ and $\mathrm{\Delta }\tau$, where the EOO parameters are $\beta =1.35$ and $\gamma =0.5$.

Figure 12

Memory capacity of a RC system for the Narma10 task, comparing single delay (black) and five delays (blue) schemes. For the single delay case, parameters were set to $\beta =1.5,\gamma =0.25$, and ${\tau }_{\text{min}}=68.4$. For the five delays case, parameters were set to $\beta =1.35, \gamma =0.5, {\tau }_{\text{min}}=40$, and $\mathrm{\Delta }\tau =79.8$. These parameters were chosen because they provide the lowest error on the respective Narma10 prediction tasks.

Figure 13

The Narma10 prediction error of a RC system for different reservoir configurations. (a) Prediction error for a single delay case ($M=1$) and $\tau ={\tau }_{\text{min}}=68.4$. (b) Prediction error for a five delay case ($M=5$) with $\mathrm{\Delta }\tau =1.6$, and ${\tau }_{\text{min}}=68.4$. (c) Prediction error for five delays ($M=5$), $\mathrm{\Delta }\tau =79.8$, and ${\tau }_{\text{min}}=40$. In each panel, the performance is shown for different input scaling and feedback strength values.