Machine-Learning Optimized Measurements of Chaotic Dynamical Systems via the Information Bottleneck

Deterministic chaos permits a precise notion of a “perfect measurement” as one that, when obtained repeatedly, captures all of the information created by the system’s evolution with minimal redundancy. Finding an optimal measurement is challenging and has generally required intimate knowledge of the dynamics in the few cases where it has been done. We establish an equivalence between a perfect measurement and a variant of the information bottleneck. As a consequence, we can employ machine learning to optimize measurement processes that efficiently extract information from trajectory data. We obtain approximately optimal measurements for multiple chaotic maps and lay the necessary groundwork for efficient information extraction from general time series.

Figure 1

(a) The attractor of the Ikeda map and a partial trajectory. States colored blue (orange) are measured as outcome $A$ ($B$). The proposed method optimizes the measurement of a chaotic system using artificial neural networks that apply an IB and vector quantization (VQ) to each continuous-valued state. (b) The space of possible discrete measurements visualized in terms of the entropy of a single measurement $H(U)$ and the rate of entropy production in the infinite duration limit $h_{\infty }(U)$. The entropy rate of any partition is upper bounded by $H(U)$ and the metric entropy $h_{\mathrm{KS}}$ (dashed lines). The blue point corresponds to the partition in (a) that was optimized with the proposed method. The black points correspond to measurements parametrized by neural networks with random weights. Error bars on the entropy rates are within the markers.

Figure 2

The fractional deviation of the entropy rate $h_{\infty }(U)$ from the metric entropy $h_{\mathrm{KS}}$ as a function of the length of the trajectory $L$ used for training for (a) Ikeda and (b) Hénon maps with standard parameters and (c) the logistic map with $r=3.7115$. The distribution of values of the entropy rate over 20 trials for each $L$ are shown as violin plots, with the extrema and median indicated by the black horizontal marks; the dotted line indicates $h_{\infty }(U)=0.99h_{\mathrm{KS}}$. For each system, the partition found with the largest entropy rate for $L=12$ is shown on the right. For the logistic map, the probability density of states $p(x)$ is colored according to the optimized partition.

Figure 3

(a) For optimized partitions of the Ikeda map, trained with sequences of length $L=12$ and with a base information annealing rate $\dot{\beta }={\dot{\beta }}_0$, the reference state (i.e., the one predicted from the sequence of measurements) strongly influences the found partition. Each displayed partition is the one with the largest entropy rate (listed under the partition) after ten trials. Coordinate axes have been suppressed. (b) The same as in (a), but with half the rate of information annealing $\dot{\beta }=0.5{\dot{\beta }}_0$ during training. (c) Two of the partitions of the Hénon map, identified with the same annealing rate as in (a). (d) Same as in (c) for the logistic map with $r=3.7115$. (e) Colorings of the Ikeda attractor where, given a sequence of three measurements ${\mathit{u}}_{:3}$, the reference state is the final state $x_3$, utilizing for the measurement $U$ the partition labeled (i) (left) and (ii) (right) in (a) and (b), respectively.