Learning by mistakes in memristor networks

Recent results revived the interest in the implementation of analog devices able to perform brainlike operations. Here we introduce a training algorithm for a memristor network which is inspired by previous work on biological learning. Robust results are obtained from computer simulations of a network of voltage-controlled memristive devices. Its implementation in hardware is straightforward, being scalable and requiring very little peripheral computation overhead.

Figure 1

(a) Example of a three-layer network with three input nodes, four intermediary nodes, and three output nodes. Each input node has a synaptic connection to every intermediary node, and each intermediary node has a similar connection to every output node. (b)–(d) Typical run to learn the six simple input-output patterns (i.e., maps) for a network of six input neurons, 300 middle neurons, and six output neurons. As noted in (b), the error eventually reaches zero, after which learning of a new pattern (c) is attempted. The list of maps is shown in (d). (Figure has been redrawn from [3].)

Figure 2

Memristor behavior as a function of resistance and applied voltage. The resistance of the memristor does not change unless the absolute value of the applied voltage is greater than $V_{\updownarrow }$. Here we used $V_{\updownarrow }=0.075$, $R_{\text{min}}=75$, $R_{\text{max}}=5000$, and $\beta =0.9$

Figure 3

Behavior of a voltage-controlled memristor connected to the circuit shown in (a). (b) Typical changes in the properties of the memristor as a function of a time-dependent voltage $V$. Also shown are (c) the resulting current $I$ and (d) the instantaneous resistance $R$. (e) The same data are presented as an $I\text{−}V$ curve. Notice that relatively small voltage excursions (i.e., upward triangular sweeps) do not change the device resistance, while relatively large voltage excursions do, resulting in the typical hysteresis loop. The results in (b)–(e) have been colored to aid the interpretation.

Figure 4

Sketch of the learning algorithm for a network with $N_{\text{in}}=2$, $N_{\text{bulk}}=4$, and $N_{\text{out}}=2$. In the reading step (a) a relatively small $V_{\text{read}}$ voltage is applied and the current at each output node is measured by the ammeter. The output node with the largest current is defined as the output. If that output is not the desired one, in the correction step (b) a relatively large $V_{\text{write}}$ voltage is applied to alter the resistance of the memristor path. The cycle is repeated until the desired map is learned.

Figure 5

Learning performance as a function of the middle layer size. Results show the success as a fraction of networks that learn a random input-output association map after a given number of correction steps for (a) $N_{\text{in}}=N_{\text{out}}=3$ and $N_{\text{bulk}}=20$, 100, and 400 and (b) $N_{\text{in}}=N_{\text{out}}=4$ and $N_{\text{bulk}}=70$, 200, and 600. Also shown is the success at 1000 steps as a function of $N_{\text{bulk}}$ for (c) $N_{\text{in}}=N_{\text{out}}=3$ and (d) $N_{\text{in}}=N_{\text{out}}=4$. In (c) and (d) the colored symbols correspond to the results in (a) and (b) obtained with the respective $N_{\text{bulk}}$ values. All results are averages over at least 500 network realizations, yielding values of standard errors smaller than the symbols size (standard error of the mean approximately equal to $0.025$).

Figure 6

Typical evolution of the training of a network learning seven successive maps for $N_{\text{in}}=N_{\text{out}}=4$, $N_{\text{bulk}}=200$, and the other parameters the same as in Fig. 5. The upper main panel shows the error (Hamming distance from the desired to the current output) as a function of training steps. (a)–(f) The input-output maps are depicted and presented sequentially as indicated in the lower main panel.

Figure 7

Example of the network recovery after a single perturbation for $N_{\text{in}}=N_{\text{out}}=4$, $N_{\text{bulk}}=200$, and the other parameters the same as in Fig. 5. We plot the error (i.e., the Hamming distance from the desired to the current output) as a function of the training steps for a network which is learning the identity map. One hundred steps after the map is learned, $10\%$ randomly chosen memristors are perturbed by increasing their resistances by $5\%$. The first downward arrow indicates the first perturbation, while the second downward arrow denotes the 14th perturbation (which did not increase the network error).

Figure 8

Evolution of the distribution of resistance values after repeated relearning of the same set of maps for a network with $N_{\text{in}}=N_{\text{out}}=3$, $N_{\text{bulk}}=400$, and other parameters values as in Fig. 5. (a) Resistance histogram at the beginning of the simulation, after learning all possible maps once (labeled 27), after shuffling and relearning all maps four times (labeled 108), and after shuffling and relearning all maps ten times (labeled 270). The dashed vertical line shows the minimum possible value $R_{\text{min}}=50$. (b) Average resistance as a function of the number of learned maps. Arrows indicate the maps used for the histograms in (a). (c) Same data as in (a) after normalizing each memristor's resistance by the average value of the whole network. (d) Coefficient of variation of the memristor's resistance as a function of the number of learned maps. Results are calculated for a single network and then averaged over 25 network realizations. The bin size (a) $\mathrm{\Delta }R=5$ and (b) $\mathrm{\Delta }R=0.05R_i$.

Figure 9

Performance of the approach using a random distribution of memristor polarities or random $V_{\text{write}}$ correction values. (a) Success as a function of step number for networks where memristors polarities are chosen at random . (b) Success as a function of step number for equal initial conditions $R_{\text{min}}=100$ and correction voltages $V_{\text{write}}$ uniformly drawn from 0.15 to 0.3. Also shown is the initial and final resistance density distribution for the case of using (c) random polarity and (d) random correction $V_{\text{write}}$. For comparison, in (a) and (b) the dashed line reproduces the results presented in Fig. 4 ($N_B=200$, black circles). In (c) and (d), the bin width $\mathrm{\Delta }R=5$. Results are averaged over at least 100 network realizations using, in all cases, $N_{\text{in}}=N_{\text{out}}=4$ and $N_{\text{bulk}}=200$.

Figure 10

The BCM behavior as a function of resistance and applied voltage, using the parameters proposed in Ref. [12]: $D=10\text{nm}$, $\mu ={10}^{-16}{\text{m}}^2{\text{V}}^{-1}{\text{s}}^{-1}$, $R_{\text{min}}={10}^3\mathrm{\Omega }$, $R_{\text{max}}={10}^4\mathrm{\Omega }$, $a=0.1494$, $b=1.6182$, $v_{t0}=0.915\text{V}$, $v_{t1}=1.3048\text{V}$, $v_{th0}=4.7404\text{V}$, and $v_{th1}=2.4629\text{V}$.

Figure 11

Learning performance as a function of the middle layer size for BCMs. Results show the success as the fraction of networks that learn a random input-output association map after a given number of correction steps for (a) $N_{\text{in}}=N_{\text{out}}=3$ and $N_{\text{bulk}}=20$, 100, and 400 and (b) $N_{\text{in}}=N_{\text{out}}=4$ and $N_{\text{bulk}}=70$, 200, and 600. Also shown is the success at 1000 steps as a function of $N_{\text{bulk}}$ for (c) $N_{\text{in}}=N_{\text{out}}=3$ and (d) $N_{\text{in}}=N_{\text{out}}=4$. In (c) and (d) the colored symbols correspond to the results in (a) and (b) obtained with the respective $N_{\text{bulk}}$ values.