Memory Formation in Adaptive Networks

The continuous adaptation of networks like our vasculature ensures optimal network performance when challenged with changing loads. Here, we show that adaptation dynamics allow a network to memorize the position of an applied load within its network morphology. We identify that the irreversible dynamics of vanishing network links encode memory. Our analytical theory successfully predicts the role of all system parameters during memory formation, including parameter values which prevent memory formation. We thus provide analytical insight on the theory of memory formation in disordered systems.

Figure 1

Networks with $\gamma =\frac{1}{2}$ undergo memory formation and retrieval. (a) Network evolved under writing stimulus which is additional load $q^{\mathrm{add}}$, equally distributed over outer nodes around ${\theta }_1$ (blue filled), for training time $t_{\text{train}}$. Outlet node depicted in red. (b) Network adapted without stimulus over waiting time $t_{\mathrm{wait}}$. Memory is subsequently probed by applying probing stimulus at angle ${\theta }_2$ (empty blue). (c) Power loss $E$ over 200 independent simulations versus ${\theta }_2-{\theta }_1$ for varying ${\theta }_2$ for trained dataset (red) and for untrained data set (blue). (d) Memory read-out signal $S$ collapses when plotted using Eq. (3) for all $t_{\text{train}}$ shown by the colorbar. (${\tau }_{\mathrm{mem}}=29\delta t$, ${\tau }_{\mathrm{cor}}=7.5\delta t$). $q^{(0)}=1$, $q^{\mathrm{add}}=2000q^{(0)}$, $N=1945$, and $T=30\delta t$.

Figure 2

(a) Ratio of conductance of two subsequent iterations versus preceding conductance during adaptation for $3t_{\text{train}}$ iterations after training phase of duration $t_{\text{train}}$ ended. Above threshold conductance $C_{\mathrm{th}}$ (vertical red dashed line) conductances fluctuate around $[C(t+\delta t)/C(t)]=1$ (horizontal red dashed line). Low conductance links follow a power law with exponent $1/3$ (red line). Only threshold conductance $C_{\mathrm{th}}$ is stimulus strength specific; compare gray ($q^{\mathrm{add}}=40000q^{(0)}$) and color ($q^{\mathrm{add}}=0$). (b) A network adapted for $t_{\text{train}}$, iterating for longer, $4t_{\text{train}}$, links with conductance smaller than threshold $C_{\mathrm{th}}$ disappear (c). $\gamma =1/2$, $q^{(0)}=1$, $N=526$, and $T=30\delta t$.

Figure 3

Optimized networks for analytical calculations. (a) Simplest network, with one link (dotted line) of vanishingly small conductance. Nodes 2 and 3 bear positive load balanced by outlet node 1 (red dot). (b) General asymmetric network, where all nodes of the upper layers are connected to node 3. $q^{(0)}$ mean fluctuating load on all the nodes, blue dots denote stimulus of additional load $q^{\mathrm{add}}$ distributed among those nodes.

Figure 4

Numerically determined threshold conductance $C_{\mathrm{th}}$ of disk-shaped networks (inset) follows analytical prediction on model parameters; compare Eq. (7). $\gamma =1/2$,$T=30\delta t$.

Figure 5

Memory formation is absent when the cost function scales with $\gamma =1$. In this case, all links are stable and the power loss $E$ in trained and untrained network is identical (inset). Same protocol as in Fig. 1.

Figure 6

Measure of vanishing link orientation for spatial signature of memory. Example network, highlighting vanishing links in light blue. Other links’ width is scaled by their conductance value, and the outlet node is depicted in red. On network enlargement, a link orientation angle ${\varphi }_{ij}$ (dark blue dotted curve) is indicated as the angle between the orientation vector $\overrightarrow{X_{ij}^0}$ of link $ij$ and the location vector of the center of the considered link $ij$ with respect to the outlet in the network ($\overrightarrow{X_{ij}^l}$).

Figure 7

Minimal conductance links retain spatial imprint of stimulus in the network morphology. (a) Example network evolved without any stimulus. Light blue links in the network represent links with minimal conductance. (b) Example network evolved with a stimulus equally distributed over the blue filled nodes. (c),(d) Average fraction of weak links at rim of width $\mathrm{\Delta }r$ of the network as a function of polar angle (c) for networks without stimuli and (d) for networks evolved with stimulus applied in (b). Data for various $\mathrm{\Delta }r$, measured in terms of the average distance, $LS$, between neighboring nodes, is shown as an average of 100 runs. (e), (f) Average angle between orientation vector and location vector (with respect to the center of the network) of minimal conductance links evaluated over a discrete grid (e) for networks without any stimulus and (f) for networks evolved with stimulus applied in (b). Averages are over 80 independent runs. Additional model parameters: $q^{\mathrm{add}}=2000q^{(0)}$, $q^{(0)}=1$, $N=1945$, $t=25\delta t$, and $\gamma =0.5$.