Criticality in reservoir computer of coupled phase oscillators

Accumulating evidence shows that the cerebral cortex is operating near a critical state featured by power-law size distribution of neural avalanche activities, yet evidence of this critical state in artificial neural networks mimicking the cerebral cortex is still lacking. Here we design an artificial neural network of coupled phase oscillators and, by the technique of reservoir computing in machine learning, train it for predicting chaos. It is found that when the machine is properly trained, oscillators in the reservoir are synchronized into clusters whose sizes follow a power-law distribution. This feature, however, is absent when the machine is poorly trained. Additionally, it is found that despite the synchronization degree of the original network, once properly trained, the reservoir network is always developed to the same critical state, exemplifying the “attractor” nature of this state in machine learning. The generality of the results is verified in different reservoir models and by different target systems, and it is found that the scaling exponent of the distribution is independent of the reservoir details and the bifurcation parameters of the target system, but is modified when the dynamics of the target system is changed to a different type. The findings shed light on the nature of machine learning, and are helpful to the design of high-performance machines in physical systems.

Figure 1

The collective behavior of the reservoir network in the absence of input. (a) $r$ versus $\alpha$. The gray zone, $\alpha \in (0.65,0.71)$, denotes the transition regime. The values of $r$ at $\alpha =0.25$, $0.53$, $0.7$, and $0.8$ are about $0.046$, $0.071$, $0.322$, and $0.952$, respectively. (b) For $\alpha =0.53$, typical states observed in the system evolution. Upper panels: the correlation matrices. Lower panels: the synchronization patterns. (c) The size distribution of the synchronization clusters with different coupling strengths.

Figure 2

With $\alpha =0.53$, the prediction performance and pattern properties of the properly (${\text{RC}}_g$) and poorly (${\text{RC}}_b$) trained machines. (a) The state evolution of the Lorenz chaotic oscillator predicted by ${\text{RC}}_g$. (b) The return map of the variable $z$ predicted by ${\text{RC}}_g$. (c) The size distribution of the synchronization clusters in the predicting phase for ${\text{RC}}_g$. Gray squares: the results for the original network without input. Red disks: the results for the reservoir network of ${\text{RC}}_g$. The results within $s\in [1,70]$ are fitted by a power-law scaling $p_s\propto s^{\gamma }$, with $\gamma \approx -1.7$. (d) The state evolution of the chaotic Lorenz oscillator predicted by ${\text{RC}}_b$. (e) The return map predicted by ${\text{RC}}_b$. (f) The size distribution of the synchronization clusters for ${\text{RC}}_b$.

Figure 3

The scaling property of the synchronization clusters in a properly trained machine with different parameters $\alpha$, $\beta$, and $\lambda$ (see Supplemental Material [36] for the parameters). For $s\in [1,70]$, all distributions follow the same power-law scaling $p_s\propto s^{\gamma }$, with $\gamma \approx -1.7$.

Figure 4

The impacts of the reservoir model, target dynamics, and RC parameters on criticality. (a1) Synchronization transition for the conventional Kuramoto model. (a2) By $\alpha =0.5$, the size distribution of the synchronization clusters for the original network (gray filled disks), the properly trained machine ${\text{RC}}_g$ (red filled disks), and the poorly trained machine ${\text{RC}}_b$ (blue open circles). (b1) For the chaotic Rössler oscillator, the state evolution predicted by ${\text{RC}}_g$. The prediction horizon is about 3 Lyapunov times. (b2) The size distribution of the synchronization clusters for ${\text{RC}}_g$. (c) The results for ${\text{RC}}_b$. (d) For the chaotic Rössler oscillator, the criticality is independent on the RC parameters (see Supplemental Material [36] for the parameters).