Energy-Efficient Stochastic Computing with Superparamagnetic Tunnel Junctions

Superparamagnetic tunnel junctions (SMTJs) have emerged as a competitive, realistic nanotechnology to support novel forms of stochastic computation in CMOS-compatible platforms. One of their applications is to generate random bitstreams suitable for use in stochastic computing implementations. We describe a method for digitally programmable bitstream generation based on precharge sense amplifiers. This generator is significantly more energy efficient than SMTJ-based bitstream generators that tune probabilities with spin currents and a factor of 2 more efficient than related CMOS-based implementations. The true randomness of this bitstream generator allows us to use them as the fundamental units of a novel neural network architecture. To take advantage of the potential savings, we codesign the algorithm with the circuit, rather than directly transcribing a classical neural network into hardware. The flexibility of the neural network mathematics allows us to adapt the network to the explicitly energy-efficient choices we make at the device level. The result is a convolutional neural network design operating at approximately $150$ nJ per inference with $97\mathrm{\%}$ performance on the MNIST data set—a factor of 1.4 to 7.7 improvement in energy efficiency over comparable proposals in the recent literature.

Figure 1

SMTJ readout. (a) The PCSA circuit for reading an SMTJ state [35]. (b) The stochastic computing PCSA (SCPCSA) includes a set-reset (SR) latch on top of the PCSA from (a), to prevent the precharging of nodes $e$ and $f$ from affecting the output duty cycle. The left and right ($e$ and $f$) branches of the latch are directly wired to the $e$ and $f$ nodes of the PSCA. A more pedagogical version of this circuit is found in Fig. 10.

Figure 2

Circuit simulations of the SCPCSA circuit. (a) Time-series resistance of the SMTJ due to thermal fluctuations. (b) Sampling clock (“clk” from Fig. 1). (c) Voltage at node $f$ from Fig. 1, the output of the standard PCSA circuit. (d) Voltage at node OUT from Fig. 1, the output of the SC PSCA.

Figure 3

Autocorrelation under a lag of one clock cycle as a function of clock cycle time $t_{\mathrm{clock}}$ compared to the SMTJ mean dwell time $\tau$. Each point gives the mean autocorrelation time over ${10}^3$ trials, with each trial sampling the SMTJ for ${10}^3$ clock cycles. The vertical error bars give 95% confidence intervals on the mean.

Figure 4

Bitstream generator with four programmable bits. Each row is a recursive subdivision unit. Assuming input probabilities of $1/2$ from the SCPCSAs, the topmost two-input multiplexer outputs $1/4$ or $3/4$ depending on $b_2$; the middle multiplexer $1/8$, $3/8$, $5/8$, and $7/8$, depending on $b_2$ and $b_3$; and so on. SRAM control-and-decode circuits, which set the bits $\overrightarrow{b}$ appropriately for the desired output probability, are hidden for clarity.

Figure 5

Energy efficiency comparison between our programmable bitstream generator (PBS) at supply voltages of 1 V and 0.8 V and a traditional LFSR with binary comparator. Plotted horizontally is the energy needed to produce a single new element of the output stochastic bitstream. The $N$-bit PBS uses $N$ SMTJs where the $N$-bit LFSR uses an $N$-bit register, ensuring a fair equiprecision comparison.

Figure 6

Output probability distributions for the 4-bit generator induced by imprecision in SMTJ fabrication. The three different standard deviations correspond to the three distributions in Fig. 11. The width of a bubble at a particular vertical coordinate gives the probability density that the generator outputs that probability in its bitstream. Note that the 3-, 2-, and 1-bit versions can be extracted by simply restricting this plot to outputs with mean values that are multiples of $1/8$, $1/4$, and $1/2$, respectively.

Figure 7

Activation pseudofunction of the or gate in terms of its bounds. The shaded region indicates possible (multivalued) outputs as a function of total input probability, in the large fan-in case. For comparison, the dashed line plots the hyperbolic tangent function, a common neural activation in deep learning applications. Like the hyperbolic tangent, the or-gate operation is asymptotically linear near the origin and saturates exponentially to unity.

Figure 8

Evaluation of our network performance for various hyperparameters. The upper curve shows variation in $\delta$, the maximum number of isolators decorrelating each of the $L$ hidden layers. This affects the energy by increasing the total warm-up time, $L\delta$, needed before network output becomes meaningful. The lower curve tracks $N$, the number of meaningful data points collected after the warm-up period concludes.

Figure 9

Comparison of stochastic computing implementations of LeNet5 on the MNIST data set from the literature. The survey of logistic, rectified linear unit (ReLU), and tanh networks is reported in Ref. [70]. Two best-performing examples are extracted for stochastic computing deep convolutional neural network (SC-DCNN) [66], and HEIF is reported in Ref. [67]. These references are all reported at the 45-nm technology node. The results for our work are presented at 1 V with $N=128$ and $\delta =16$ and are scaled up by a factor of 4 to bring our 22-nm node calculation into fair comparison with the literature.

Figure 10

Operation of SCPCSA. Blue wires represent $V=V_{dd}$, red wires represent $V=0$, and red circles indicate that the source-drain path of the transistor is turned off. (a) Steady-state conclusion of evaluation phase, corresponding to the far right of (f). The PCSA has detected that the SMTJ is in the parallel state, pulling node $f$ to ground and node $e$ to $V_{dd}$ as a result. (b) The SR latch copies the value of node $f$ to the output node. (c) Steady-state conclusion of the precharge phase, corresponding to the far right of (g). (d) The PCSA has brought both $e$ and $f$ to $V_{dd}$ in preparation for the measurement in the next evaluation phase. Although $f$ has been brought to $V_{dd}$, the internal state of the latch does not change, so the output node continues to correctly represent the previously measured state of the SMTJ. (e)–(g) States of the various circuit node voltages throughout a simulation of the SCPCSA. Curve colors correspond to points in the circuit indicated by the small circles in (a)–(d).

Figure 11

Probability density functions for the beta distribution centered at $1/2$ with three different standard deviations.

Figure 12

(a) Schematic block diagram of LeNet5 neural network. (b) Schematic block diagram of a neural layer (either a convolutional or a fully connected layer). Each of these layers contains excitatory and inhibitory subnetworks. (c) Schematic view of a functional neural block comprised of SMTJs, and gates, one or gate, and a decorrelator.