Limits...
Back-propagation operation for analog neural network hardware with synapse components having hysteresis characteristics.

Ueda M, Nishitani Y, Kaneko Y, Omote A - PLoS ONE (2014)

Bottom Line: We examined the probability that the error decreased to a designated value within a predetermined loop number.The ferroelectric has the characteristics that the magnitude of polarization does not become smaller when voltages having the same polarity are applied.These characteristics greatly improved the probability even if the learning rate was small, if the magnitude of the dispersion is adequate.

View Article: PubMed Central - PubMed

Affiliation: Advanced Research Division, Panasonic Corporation, Soraku, Kyoto, Japan.

ABSTRACT
To realize an analog artificial neural network hardware, the circuit element for synapse function is important because the number of synapse elements is much larger than that of neuron elements. One of the candidates for this synapse element is a ferroelectric memristor. This device functions as a voltage controllable variable resistor, which can be applied to a synapse weight. However, its conductance shows hysteresis characteristics and dispersion to the input voltage. Therefore, the conductance values vary according to the history of the height and the width of the applied pulse voltage. Due to the difficulty of controlling the accurate conductance, it is not easy to apply the back-propagation learning algorithm to the neural network hardware having memristor synapses. To solve this problem, we proposed and simulated a learning operation procedure as follows. Employing a weight perturbation technique, we derived the error change. When the error reduced, the next pulse voltage was updated according to the back-propagation learning algorithm. If the error increased the amplitude of the next voltage pulse was set in such way as to cause similar memristor conductance but in the opposite voltage scanning direction. By this operation, we could eliminate the hysteresis and confirmed that the simulation of the learning operation converged. We also adopted conductance dispersion numerically in the simulation. We examined the probability that the error decreased to a designated value within a predetermined loop number. The ferroelectric has the characteristics that the magnitude of polarization does not become smaller when voltages having the same polarity are applied. These characteristics greatly improved the probability even if the learning rate was small, if the magnitude of the dispersion is adequate. Because the dispersion of analog circuit elements is inevitable, this learning operation procedure is useful for analog neural network hardware.

Show MeSH
Schematics of FeMEM and its electrical properties.The fabricated FeMEM showed (a) electron gas accumulation and (b) complete depletion switching operation due to the reversal of ferroelectric polarization. (c) GF–VG characteristics showed the counterclockwise hysteresis loop corresponding to the switching of ferroelectric polarization.
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4231062&req=5

pone-0112659-g003: Schematics of FeMEM and its electrical properties.The fabricated FeMEM showed (a) electron gas accumulation and (b) complete depletion switching operation due to the reversal of ferroelectric polarization. (c) GF–VG characteristics showed the counterclockwise hysteresis loop corresponding to the switching of ferroelectric polarization.

Mentions: We fabricated a FeMEM structure based on insights gained in previous studies [12]–[14]. As shown in Figure 3(a) (b), the FeMEM consists of a semiconductor film of ZnO, a ferroelectric film of Pb(Zr,Ti)O3 (PZT), and a bottom gate electrode of SrRuO3 (SRO). All the layers of ZnO/PZT/SRO were epitaxially grown over a SrTiO3 (STO) substrate by pulsed laser deposition. Pt/Ti electrodes were used for the source and drain contacts to the ZnO film. The fabricated FeMEM showed electron gas accumulation and complete depletion switching operation due to reversal of the ferroelectric polarization. The channel conductance (GF)–gate voltage (VG) characteristics of the FeMEM are shown in Figure 3(c). The GF–VG characteristics were measured using a semiconductor parametric analyzer (Agilent 4155C) under the condition of long integration time. By measuring the drain current under the condition of drain voltage = 0.1 V, GF was calculated. The drain voltage was set to be low so as not to change the polarization of the ferroelectric. The figure shows counterclockwise hysteresis loops corresponding to the switching of ferroelectric polarization. The conductance at VG = 0 V changed according to the history of applied VG and could thus take multiple values. It was confirmed that there was no notable degradation of conductance over 105 s [12]. These characteristics allowed the construction of an analog ANN circuit with synapse elements using the FeMEM [15]–[18], [21].


Back-propagation operation for analog neural network hardware with synapse components having hysteresis characteristics.

Ueda M, Nishitani Y, Kaneko Y, Omote A - PLoS ONE (2014)

Schematics of FeMEM and its electrical properties.The fabricated FeMEM showed (a) electron gas accumulation and (b) complete depletion switching operation due to the reversal of ferroelectric polarization. (c) GF–VG characteristics showed the counterclockwise hysteresis loop corresponding to the switching of ferroelectric polarization.
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4231062&req=5

pone-0112659-g003: Schematics of FeMEM and its electrical properties.The fabricated FeMEM showed (a) electron gas accumulation and (b) complete depletion switching operation due to the reversal of ferroelectric polarization. (c) GF–VG characteristics showed the counterclockwise hysteresis loop corresponding to the switching of ferroelectric polarization.
Mentions: We fabricated a FeMEM structure based on insights gained in previous studies [12]–[14]. As shown in Figure 3(a) (b), the FeMEM consists of a semiconductor film of ZnO, a ferroelectric film of Pb(Zr,Ti)O3 (PZT), and a bottom gate electrode of SrRuO3 (SRO). All the layers of ZnO/PZT/SRO were epitaxially grown over a SrTiO3 (STO) substrate by pulsed laser deposition. Pt/Ti electrodes were used for the source and drain contacts to the ZnO film. The fabricated FeMEM showed electron gas accumulation and complete depletion switching operation due to reversal of the ferroelectric polarization. The channel conductance (GF)–gate voltage (VG) characteristics of the FeMEM are shown in Figure 3(c). The GF–VG characteristics were measured using a semiconductor parametric analyzer (Agilent 4155C) under the condition of long integration time. By measuring the drain current under the condition of drain voltage = 0.1 V, GF was calculated. The drain voltage was set to be low so as not to change the polarization of the ferroelectric. The figure shows counterclockwise hysteresis loops corresponding to the switching of ferroelectric polarization. The conductance at VG = 0 V changed according to the history of applied VG and could thus take multiple values. It was confirmed that there was no notable degradation of conductance over 105 s [12]. These characteristics allowed the construction of an analog ANN circuit with synapse elements using the FeMEM [15]–[18], [21].

Bottom Line: We examined the probability that the error decreased to a designated value within a predetermined loop number.The ferroelectric has the characteristics that the magnitude of polarization does not become smaller when voltages having the same polarity are applied.These characteristics greatly improved the probability even if the learning rate was small, if the magnitude of the dispersion is adequate.

View Article: PubMed Central - PubMed

Affiliation: Advanced Research Division, Panasonic Corporation, Soraku, Kyoto, Japan.

ABSTRACT
To realize an analog artificial neural network hardware, the circuit element for synapse function is important because the number of synapse elements is much larger than that of neuron elements. One of the candidates for this synapse element is a ferroelectric memristor. This device functions as a voltage controllable variable resistor, which can be applied to a synapse weight. However, its conductance shows hysteresis characteristics and dispersion to the input voltage. Therefore, the conductance values vary according to the history of the height and the width of the applied pulse voltage. Due to the difficulty of controlling the accurate conductance, it is not easy to apply the back-propagation learning algorithm to the neural network hardware having memristor synapses. To solve this problem, we proposed and simulated a learning operation procedure as follows. Employing a weight perturbation technique, we derived the error change. When the error reduced, the next pulse voltage was updated according to the back-propagation learning algorithm. If the error increased the amplitude of the next voltage pulse was set in such way as to cause similar memristor conductance but in the opposite voltage scanning direction. By this operation, we could eliminate the hysteresis and confirmed that the simulation of the learning operation converged. We also adopted conductance dispersion numerically in the simulation. We examined the probability that the error decreased to a designated value within a predetermined loop number. The ferroelectric has the characteristics that the magnitude of polarization does not become smaller when voltages having the same polarity are applied. These characteristics greatly improved the probability even if the learning rate was small, if the magnitude of the dispersion is adequate. Because the dispersion of analog circuit elements is inevitable, this learning operation procedure is useful for analog neural network hardware.

Show MeSH