Limits...
Towards a general theory of neural computation based on prediction by single neurons.

Fiorillo CD - PLoS ONE (2008)

Bottom Line: To minimize the error in its predictions and to respond only when excitation is "new and surprising," the neuron selects amongst its prior information sources through an anti-Hebbian rule.The unique inputs of a mature neuron would therefore result from learning about spatial and temporal patterns in its local environment, and by extension, the external world.Thus the theory describes how the structure of the mature nervous system could reflect the structure of the external world, and how the complexity and intelligence of the system might develop from a population of undifferentiated neurons, each implementing similar learning algorithms.

View Article: PubMed Central - PubMed

Affiliation: Department of Neurobiology, Stanford University, Stanford, California, USA. chris@monkeybiz.stanford.edu

ABSTRACT
Although there has been tremendous progress in understanding the mechanics of the nervous system, there has not been a general theory of its computational function. Here I present a theory that relates the established biophysical properties of single generic neurons to principles of Bayesian probability theory, reinforcement learning and efficient coding. I suggest that this theory addresses the general computational problem facing the nervous system. Each neuron is proposed to mirror the function of the whole system in learning to predict aspects of the world related to future reward. According to the model, a typical neuron receives current information about the state of the world from a subset of its excitatory synaptic inputs, and prior information from its other inputs. Prior information would be contributed by synaptic inputs representing distinct regions of space, and by different types of non-synaptic, voltage-regulated channels representing distinct periods of the past. The neuron's membrane voltage is proposed to signal the difference between current and prior information ("prediction error" or "surprise"). A neuron would apply a Hebbian plasticity rule to select those excitatory inputs that are the most closely correlated with reward but are the least predictable, since unpredictable inputs provide the neuron with the most "new" information about future reward. To minimize the error in its predictions and to respond only when excitation is "new and surprising," the neuron selects amongst its prior information sources through an anti-Hebbian rule. The unique inputs of a mature neuron would therefore result from learning about spatial and temporal patterns in its local environment, and by extension, the external world. Thus the theory describes how the structure of the mature nervous system could reflect the structure of the external world, and how the complexity and intelligence of the system might develop from a population of undifferentiated neurons, each implementing similar learning algorithms.

Show MeSH

Related in: MedlinePlus

Selection of ion channels by plasticity rules.Hebbian (equation 4) and anti-Hebbian (equation 3) rules selected the channels of layers 1 and 2, respectively, in a single-compartment, graded-potential, Hodgkin-Huxley-type model neuron. See Text S1 for details. Initially, there were a total of 800 glutamate-gated non-selective cation channels in layer 1 (evenly divided among 4 subtypes) and 800 voltage-gated K+ channels in layer 2 (evenly divided among 9 subtypes). Other simulations began with different numbers and proportions of channels (not shown). The final numbers of channels were the same in all cases, regardless of the starting numbers (Table S1). A. At each time step, glutamate concentration was drawn from a Gaussian distribution with a standard deviation of 20% of the mean. The mean concentration increased from 50 to 1000 µM for 10 time steps starting at 2000, and again for 500 time steps starting at 2200. After 5000 time steps the pattern repeated, for a total of 20,000 cycles. B. Membrane voltage is shown for the first and last cycles. The average membrane voltage shifted towards the  point (θ = −50 mV) of the plasticity algorithms (equations 3 and 4). The hyperpolarization starting at 2000 in the last cycle was caused by activation of “type 2” K+ channels (see panel G) triggered by the first glutamate-driven depolarization. In the first cycles, these K+ channels were not activated because the first glutamate-driven depolarization was not large enough (see legend for panel D). The increased variance in membrane voltage in the last cycle was due to a decline in total membrane conductance together with an increased sensitivity of the glutamate-gated conductance to glutamate concentration. C. The activities (open probabilities) of the four types of glutamate-gated channel, each of which differed in its affinity for glutamate (KD). Each channel was gated by a single two-state glutamate sensor. D. The Hebbian rule (equation 4) selected primarily a glutamate receptor with moderate affinity (KD = 1000 µM, shown in blue). Elimination of high affinity receptors that were always near saturation increased the sensitivity of membrane voltage to glutamate concentration. The resulting increase in depolarization to the first pulse of glutamate allowed for activation of “type 2” K+ channels (see panel G). E. Activities of “Type 1” K+ channels during the last cycle. Each of four channel types, differing in their kinetic properties, was gated by a single two-state voltage sensor with a half-maximal activation at −40 mV. Maximal time constants (at −40 mV) ranged from 10 to 333 time units. F. Of the K+ channels in panel E, the anti-Hebbian rule (equation 3) ultimately selected the one with the fastest kinetics. Initially, the number of each type of K+ channel increased from its starting value of 89 because the membrane voltage was almost always depolarized beyond θ (−50 mV). G. The activities of “type 2” K+ channels during the last cycle. These channels each consisted of 2 layers of 4 sensors each. The sensors of the first layer were not modeled realistically, but instead were all “turned on” instantly whenever the membrane was depolarized beyond −25 mV. They then turned off slowly. The sensors of the second layer adapted to those of the first layer with kinetics that varied across channels as shown. Each channel was open only when at least one sensor in layer 1 was on and all sensors in layer 2 were on. H. The anti-Hebbian rule selected the type 2 K+ channel with intermediate kinetics (τ = 100). This channel was able to use the first pulse of glutamate to predict and partially cancel the effect of the second pulse of glutamate. Initially, the numbers of all type 2 K+ channels declined because membrane voltage never exceeded the threshold necessary to activate them.
© Copyright Policy
Related In: Results  -  Collection


getmorefigures.php?uid=PMC2553191&req=5

pone-0003298-g003: Selection of ion channels by plasticity rules.Hebbian (equation 4) and anti-Hebbian (equation 3) rules selected the channels of layers 1 and 2, respectively, in a single-compartment, graded-potential, Hodgkin-Huxley-type model neuron. See Text S1 for details. Initially, there were a total of 800 glutamate-gated non-selective cation channels in layer 1 (evenly divided among 4 subtypes) and 800 voltage-gated K+ channels in layer 2 (evenly divided among 9 subtypes). Other simulations began with different numbers and proportions of channels (not shown). The final numbers of channels were the same in all cases, regardless of the starting numbers (Table S1). A. At each time step, glutamate concentration was drawn from a Gaussian distribution with a standard deviation of 20% of the mean. The mean concentration increased from 50 to 1000 µM for 10 time steps starting at 2000, and again for 500 time steps starting at 2200. After 5000 time steps the pattern repeated, for a total of 20,000 cycles. B. Membrane voltage is shown for the first and last cycles. The average membrane voltage shifted towards the point (θ = −50 mV) of the plasticity algorithms (equations 3 and 4). The hyperpolarization starting at 2000 in the last cycle was caused by activation of “type 2” K+ channels (see panel G) triggered by the first glutamate-driven depolarization. In the first cycles, these K+ channels were not activated because the first glutamate-driven depolarization was not large enough (see legend for panel D). The increased variance in membrane voltage in the last cycle was due to a decline in total membrane conductance together with an increased sensitivity of the glutamate-gated conductance to glutamate concentration. C. The activities (open probabilities) of the four types of glutamate-gated channel, each of which differed in its affinity for glutamate (KD). Each channel was gated by a single two-state glutamate sensor. D. The Hebbian rule (equation 4) selected primarily a glutamate receptor with moderate affinity (KD = 1000 µM, shown in blue). Elimination of high affinity receptors that were always near saturation increased the sensitivity of membrane voltage to glutamate concentration. The resulting increase in depolarization to the first pulse of glutamate allowed for activation of “type 2” K+ channels (see panel G). E. Activities of “Type 1” K+ channels during the last cycle. Each of four channel types, differing in their kinetic properties, was gated by a single two-state voltage sensor with a half-maximal activation at −40 mV. Maximal time constants (at −40 mV) ranged from 10 to 333 time units. F. Of the K+ channels in panel E, the anti-Hebbian rule (equation 3) ultimately selected the one with the fastest kinetics. Initially, the number of each type of K+ channel increased from its starting value of 89 because the membrane voltage was almost always depolarized beyond θ (−50 mV). G. The activities of “type 2” K+ channels during the last cycle. These channels each consisted of 2 layers of 4 sensors each. The sensors of the first layer were not modeled realistically, but instead were all “turned on” instantly whenever the membrane was depolarized beyond −25 mV. They then turned off slowly. The sensors of the second layer adapted to those of the first layer with kinetics that varied across channels as shown. Each channel was open only when at least one sensor in layer 1 was on and all sensors in layer 2 were on. H. The anti-Hebbian rule selected the type 2 K+ channel with intermediate kinetics (τ = 100). This channel was able to use the first pulse of glutamate to predict and partially cancel the effect of the second pulse of glutamate. Initially, the numbers of all type 2 K+ channels declined because membrane voltage never exceeded the threshold necessary to activate them.

Mentions: The simulated neuron simultaneously selected from amongst four subtypes of glutamate-gated cation channels in layer 1 through a Hebbian rule (equation 4), and from amongst nine subtypes of voltage-regulated potassium channels in layer 2 through an anti-Hebbian rule (equation 3). The stimulus was glutamate concentration, which was drawn from a Gaussian distribution at each time point (Fig. 3A). The mean concentration increased during two square wave pulses (the first of which was very brief). This pattern repeated itself for a total of 20,000 cycles. The final number of channels of each of type appeared to have little or no dependence on the starting numbers (Table S1).


Towards a general theory of neural computation based on prediction by single neurons.

Fiorillo CD - PLoS ONE (2008)

Selection of ion channels by plasticity rules.Hebbian (equation 4) and anti-Hebbian (equation 3) rules selected the channels of layers 1 and 2, respectively, in a single-compartment, graded-potential, Hodgkin-Huxley-type model neuron. See Text S1 for details. Initially, there were a total of 800 glutamate-gated non-selective cation channels in layer 1 (evenly divided among 4 subtypes) and 800 voltage-gated K+ channels in layer 2 (evenly divided among 9 subtypes). Other simulations began with different numbers and proportions of channels (not shown). The final numbers of channels were the same in all cases, regardless of the starting numbers (Table S1). A. At each time step, glutamate concentration was drawn from a Gaussian distribution with a standard deviation of 20% of the mean. The mean concentration increased from 50 to 1000 µM for 10 time steps starting at 2000, and again for 500 time steps starting at 2200. After 5000 time steps the pattern repeated, for a total of 20,000 cycles. B. Membrane voltage is shown for the first and last cycles. The average membrane voltage shifted towards the  point (θ = −50 mV) of the plasticity algorithms (equations 3 and 4). The hyperpolarization starting at 2000 in the last cycle was caused by activation of “type 2” K+ channels (see panel G) triggered by the first glutamate-driven depolarization. In the first cycles, these K+ channels were not activated because the first glutamate-driven depolarization was not large enough (see legend for panel D). The increased variance in membrane voltage in the last cycle was due to a decline in total membrane conductance together with an increased sensitivity of the glutamate-gated conductance to glutamate concentration. C. The activities (open probabilities) of the four types of glutamate-gated channel, each of which differed in its affinity for glutamate (KD). Each channel was gated by a single two-state glutamate sensor. D. The Hebbian rule (equation 4) selected primarily a glutamate receptor with moderate affinity (KD = 1000 µM, shown in blue). Elimination of high affinity receptors that were always near saturation increased the sensitivity of membrane voltage to glutamate concentration. The resulting increase in depolarization to the first pulse of glutamate allowed for activation of “type 2” K+ channels (see panel G). E. Activities of “Type 1” K+ channels during the last cycle. Each of four channel types, differing in their kinetic properties, was gated by a single two-state voltage sensor with a half-maximal activation at −40 mV. Maximal time constants (at −40 mV) ranged from 10 to 333 time units. F. Of the K+ channels in panel E, the anti-Hebbian rule (equation 3) ultimately selected the one with the fastest kinetics. Initially, the number of each type of K+ channel increased from its starting value of 89 because the membrane voltage was almost always depolarized beyond θ (−50 mV). G. The activities of “type 2” K+ channels during the last cycle. These channels each consisted of 2 layers of 4 sensors each. The sensors of the first layer were not modeled realistically, but instead were all “turned on” instantly whenever the membrane was depolarized beyond −25 mV. They then turned off slowly. The sensors of the second layer adapted to those of the first layer with kinetics that varied across channels as shown. Each channel was open only when at least one sensor in layer 1 was on and all sensors in layer 2 were on. H. The anti-Hebbian rule selected the type 2 K+ channel with intermediate kinetics (τ = 100). This channel was able to use the first pulse of glutamate to predict and partially cancel the effect of the second pulse of glutamate. Initially, the numbers of all type 2 K+ channels declined because membrane voltage never exceeded the threshold necessary to activate them.
© Copyright Policy
Related In: Results  -  Collection

Show All Figures
getmorefigures.php?uid=PMC2553191&req=5

pone-0003298-g003: Selection of ion channels by plasticity rules.Hebbian (equation 4) and anti-Hebbian (equation 3) rules selected the channels of layers 1 and 2, respectively, in a single-compartment, graded-potential, Hodgkin-Huxley-type model neuron. See Text S1 for details. Initially, there were a total of 800 glutamate-gated non-selective cation channels in layer 1 (evenly divided among 4 subtypes) and 800 voltage-gated K+ channels in layer 2 (evenly divided among 9 subtypes). Other simulations began with different numbers and proportions of channels (not shown). The final numbers of channels were the same in all cases, regardless of the starting numbers (Table S1). A. At each time step, glutamate concentration was drawn from a Gaussian distribution with a standard deviation of 20% of the mean. The mean concentration increased from 50 to 1000 µM for 10 time steps starting at 2000, and again for 500 time steps starting at 2200. After 5000 time steps the pattern repeated, for a total of 20,000 cycles. B. Membrane voltage is shown for the first and last cycles. The average membrane voltage shifted towards the point (θ = −50 mV) of the plasticity algorithms (equations 3 and 4). The hyperpolarization starting at 2000 in the last cycle was caused by activation of “type 2” K+ channels (see panel G) triggered by the first glutamate-driven depolarization. In the first cycles, these K+ channels were not activated because the first glutamate-driven depolarization was not large enough (see legend for panel D). The increased variance in membrane voltage in the last cycle was due to a decline in total membrane conductance together with an increased sensitivity of the glutamate-gated conductance to glutamate concentration. C. The activities (open probabilities) of the four types of glutamate-gated channel, each of which differed in its affinity for glutamate (KD). Each channel was gated by a single two-state glutamate sensor. D. The Hebbian rule (equation 4) selected primarily a glutamate receptor with moderate affinity (KD = 1000 µM, shown in blue). Elimination of high affinity receptors that were always near saturation increased the sensitivity of membrane voltage to glutamate concentration. The resulting increase in depolarization to the first pulse of glutamate allowed for activation of “type 2” K+ channels (see panel G). E. Activities of “Type 1” K+ channels during the last cycle. Each of four channel types, differing in their kinetic properties, was gated by a single two-state voltage sensor with a half-maximal activation at −40 mV. Maximal time constants (at −40 mV) ranged from 10 to 333 time units. F. Of the K+ channels in panel E, the anti-Hebbian rule (equation 3) ultimately selected the one with the fastest kinetics. Initially, the number of each type of K+ channel increased from its starting value of 89 because the membrane voltage was almost always depolarized beyond θ (−50 mV). G. The activities of “type 2” K+ channels during the last cycle. These channels each consisted of 2 layers of 4 sensors each. The sensors of the first layer were not modeled realistically, but instead were all “turned on” instantly whenever the membrane was depolarized beyond −25 mV. They then turned off slowly. The sensors of the second layer adapted to those of the first layer with kinetics that varied across channels as shown. Each channel was open only when at least one sensor in layer 1 was on and all sensors in layer 2 were on. H. The anti-Hebbian rule selected the type 2 K+ channel with intermediate kinetics (τ = 100). This channel was able to use the first pulse of glutamate to predict and partially cancel the effect of the second pulse of glutamate. Initially, the numbers of all type 2 K+ channels declined because membrane voltage never exceeded the threshold necessary to activate them.
Mentions: The simulated neuron simultaneously selected from amongst four subtypes of glutamate-gated cation channels in layer 1 through a Hebbian rule (equation 4), and from amongst nine subtypes of voltage-regulated potassium channels in layer 2 through an anti-Hebbian rule (equation 3). The stimulus was glutamate concentration, which was drawn from a Gaussian distribution at each time point (Fig. 3A). The mean concentration increased during two square wave pulses (the first of which was very brief). This pattern repeated itself for a total of 20,000 cycles. The final number of channels of each of type appeared to have little or no dependence on the starting numbers (Table S1).

Bottom Line: To minimize the error in its predictions and to respond only when excitation is "new and surprising," the neuron selects amongst its prior information sources through an anti-Hebbian rule.The unique inputs of a mature neuron would therefore result from learning about spatial and temporal patterns in its local environment, and by extension, the external world.Thus the theory describes how the structure of the mature nervous system could reflect the structure of the external world, and how the complexity and intelligence of the system might develop from a population of undifferentiated neurons, each implementing similar learning algorithms.

View Article: PubMed Central - PubMed

Affiliation: Department of Neurobiology, Stanford University, Stanford, California, USA. chris@monkeybiz.stanford.edu

ABSTRACT
Although there has been tremendous progress in understanding the mechanics of the nervous system, there has not been a general theory of its computational function. Here I present a theory that relates the established biophysical properties of single generic neurons to principles of Bayesian probability theory, reinforcement learning and efficient coding. I suggest that this theory addresses the general computational problem facing the nervous system. Each neuron is proposed to mirror the function of the whole system in learning to predict aspects of the world related to future reward. According to the model, a typical neuron receives current information about the state of the world from a subset of its excitatory synaptic inputs, and prior information from its other inputs. Prior information would be contributed by synaptic inputs representing distinct regions of space, and by different types of non-synaptic, voltage-regulated channels representing distinct periods of the past. The neuron's membrane voltage is proposed to signal the difference between current and prior information ("prediction error" or "surprise"). A neuron would apply a Hebbian plasticity rule to select those excitatory inputs that are the most closely correlated with reward but are the least predictable, since unpredictable inputs provide the neuron with the most "new" information about future reward. To minimize the error in its predictions and to respond only when excitation is "new and surprising," the neuron selects amongst its prior information sources through an anti-Hebbian rule. The unique inputs of a mature neuron would therefore result from learning about spatial and temporal patterns in its local environment, and by extension, the external world. Thus the theory describes how the structure of the mature nervous system could reflect the structure of the external world, and how the complexity and intelligence of the system might develop from a population of undifferentiated neurons, each implementing similar learning algorithms.

Show MeSH
Related in: MedlinePlus