Limits...
Back-propagation operation for analog neural network hardware with synapse components having hysteresis characteristics.

Ueda M, Nishitani Y, Kaneko Y, Omote A - PLoS ONE (2014)

Bottom Line: We examined the probability that the error decreased to a designated value within a predetermined loop number.The ferroelectric has the characteristics that the magnitude of polarization does not become smaller when voltages having the same polarity are applied.These characteristics greatly improved the probability even if the learning rate was small, if the magnitude of the dispersion is adequate.

View Article: PubMed Central - PubMed

Affiliation: Advanced Research Division, Panasonic Corporation, Soraku, Kyoto, Japan.

ABSTRACT
To realize an analog artificial neural network hardware, the circuit element for synapse function is important because the number of synapse elements is much larger than that of neuron elements. One of the candidates for this synapse element is a ferroelectric memristor. This device functions as a voltage controllable variable resistor, which can be applied to a synapse weight. However, its conductance shows hysteresis characteristics and dispersion to the input voltage. Therefore, the conductance values vary according to the history of the height and the width of the applied pulse voltage. Due to the difficulty of controlling the accurate conductance, it is not easy to apply the back-propagation learning algorithm to the neural network hardware having memristor synapses. To solve this problem, we proposed and simulated a learning operation procedure as follows. Employing a weight perturbation technique, we derived the error change. When the error reduced, the next pulse voltage was updated according to the back-propagation learning algorithm. If the error increased the amplitude of the next voltage pulse was set in such way as to cause similar memristor conductance but in the opposite voltage scanning direction. By this operation, we could eliminate the hysteresis and confirmed that the simulation of the learning operation converged. We also adopted conductance dispersion numerically in the simulation. We examined the probability that the error decreased to a designated value within a predetermined loop number. The ferroelectric has the characteristics that the magnitude of polarization does not become smaller when voltages having the same polarity are applied. These characteristics greatly improved the probability even if the learning rate was small, if the magnitude of the dispersion is adequate. Because the dispersion of analog circuit elements is inevitable, this learning operation procedure is useful for analog neural network hardware.

Show MeSH

Related in: MedlinePlus

Schematics of measurement setup and calculated conductance.With the measurement setup shown in (a), the conductance of the FeMEM can be calculated from the measured output voltage (Vout) from the op-amp when input voltage Vin = 0.1 V. After applying a reset pulse (VR) and a write pulse (VP) to the gate electrode of the FeMEM, Vout is measured. The calculated conductance is shown in (b). The open circles indicate the average values and the error bars indicate the standard deviation over 300 scans.
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4231062&req=5

pone-0112659-g004: Schematics of measurement setup and calculated conductance.With the measurement setup shown in (a), the conductance of the FeMEM can be calculated from the measured output voltage (Vout) from the op-amp when input voltage Vin = 0.1 V. After applying a reset pulse (VR) and a write pulse (VP) to the gate electrode of the FeMEM, Vout is measured. The calculated conductance is shown in (b). The open circles indicate the average values and the error bars indicate the standard deviation over 300 scans.

Mentions: We examined the performance of the basic neuron circuit. The experimental setup used to evaluate the relation of the pulse voltage (VP) and the conductance of the FeMEM is shown in Figure 4(a). The devices we use in this experiment have been tested before and were found to exhibit good non-volatility characteristics [12]. The pulse width of VP was set to 1 ms. To enhance the conductance repeatability, the conductance was measured after applying a reset pulse (VR). VR = −2 V when VP>0 and VR = 3 V when VP<0. VP was first increased from 0 to 3 V in 0.2 V steps and then reduced from 0 to −2 V in −0.2 V steps. In the same manner as GF–VG measurement, the drain current was measured under the condition of drain voltage = 0.1 so as not to change the polarization of the ferroelectric.


Back-propagation operation for analog neural network hardware with synapse components having hysteresis characteristics.

Ueda M, Nishitani Y, Kaneko Y, Omote A - PLoS ONE (2014)

Schematics of measurement setup and calculated conductance.With the measurement setup shown in (a), the conductance of the FeMEM can be calculated from the measured output voltage (Vout) from the op-amp when input voltage Vin = 0.1 V. After applying a reset pulse (VR) and a write pulse (VP) to the gate electrode of the FeMEM, Vout is measured. The calculated conductance is shown in (b). The open circles indicate the average values and the error bars indicate the standard deviation over 300 scans.
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4231062&req=5

pone-0112659-g004: Schematics of measurement setup and calculated conductance.With the measurement setup shown in (a), the conductance of the FeMEM can be calculated from the measured output voltage (Vout) from the op-amp when input voltage Vin = 0.1 V. After applying a reset pulse (VR) and a write pulse (VP) to the gate electrode of the FeMEM, Vout is measured. The calculated conductance is shown in (b). The open circles indicate the average values and the error bars indicate the standard deviation over 300 scans.
Mentions: We examined the performance of the basic neuron circuit. The experimental setup used to evaluate the relation of the pulse voltage (VP) and the conductance of the FeMEM is shown in Figure 4(a). The devices we use in this experiment have been tested before and were found to exhibit good non-volatility characteristics [12]. The pulse width of VP was set to 1 ms. To enhance the conductance repeatability, the conductance was measured after applying a reset pulse (VR). VR = −2 V when VP>0 and VR = 3 V when VP<0. VP was first increased from 0 to 3 V in 0.2 V steps and then reduced from 0 to −2 V in −0.2 V steps. In the same manner as GF–VG measurement, the drain current was measured under the condition of drain voltage = 0.1 so as not to change the polarization of the ferroelectric.

Bottom Line: We examined the probability that the error decreased to a designated value within a predetermined loop number.The ferroelectric has the characteristics that the magnitude of polarization does not become smaller when voltages having the same polarity are applied.These characteristics greatly improved the probability even if the learning rate was small, if the magnitude of the dispersion is adequate.

View Article: PubMed Central - PubMed

Affiliation: Advanced Research Division, Panasonic Corporation, Soraku, Kyoto, Japan.

ABSTRACT
To realize an analog artificial neural network hardware, the circuit element for synapse function is important because the number of synapse elements is much larger than that of neuron elements. One of the candidates for this synapse element is a ferroelectric memristor. This device functions as a voltage controllable variable resistor, which can be applied to a synapse weight. However, its conductance shows hysteresis characteristics and dispersion to the input voltage. Therefore, the conductance values vary according to the history of the height and the width of the applied pulse voltage. Due to the difficulty of controlling the accurate conductance, it is not easy to apply the back-propagation learning algorithm to the neural network hardware having memristor synapses. To solve this problem, we proposed and simulated a learning operation procedure as follows. Employing a weight perturbation technique, we derived the error change. When the error reduced, the next pulse voltage was updated according to the back-propagation learning algorithm. If the error increased the amplitude of the next voltage pulse was set in such way as to cause similar memristor conductance but in the opposite voltage scanning direction. By this operation, we could eliminate the hysteresis and confirmed that the simulation of the learning operation converged. We also adopted conductance dispersion numerically in the simulation. We examined the probability that the error decreased to a designated value within a predetermined loop number. The ferroelectric has the characteristics that the magnitude of polarization does not become smaller when voltages having the same polarity are applied. These characteristics greatly improved the probability even if the learning rate was small, if the magnitude of the dispersion is adequate. Because the dispersion of analog circuit elements is inevitable, this learning operation procedure is useful for analog neural network hardware.

Show MeSH
Related in: MedlinePlus