Limits...
A new framework for cortico-striatal plasticity: behavioural theory meets in vitro data at the reinforcement-action interface.

Gurney KN, Humphries MD, Redgrave P - PLoS Biol. (2015)

Bottom Line: Third, the two types of striatal output neuron have apparently opposite effects on action selection.Validating the model, we show it can account for behavioural data describing extinction, renewal, and reacquisition, and replicate in vitro experimental data on cortico-striatal plasticity.By bridging the levels between the single synapse and behaviour, our model shows how striatum acts as the action-reinforcement interface.

View Article: PubMed Central - PubMed

Affiliation: Department of Psychology, Adaptive Behaviour Research Group, University of Sheffield, United Kingdom; INSIGNEO Institute for In Silico Medicine, University of Sheffield, United Kingdom.

ABSTRACT
Operant learning requires that reinforcement signals interact with action representations at a suitable neural interface. Much evidence suggests that this occurs when phasic dopamine, acting as a reinforcement prediction error, gates plasticity at cortico-striatal synapses, and thereby changes the future likelihood of selecting the action(s) coded by striatal neurons. But this hypothesis faces serious challenges. First, cortico-striatal plasticity is inexplicably complex, depending on spike timing, dopamine level, and dopamine receptor type. Second, there is a credit assignment problem-action selection signals occur long before the consequent dopamine reinforcement signal. Third, the two types of striatal output neuron have apparently opposite effects on action selection. Whether these factors rule out the interface hypothesis and how they interact to produce reinforcement learning is unknown. We present a computational framework that addresses these challenges. We first predict the expected activity changes over an operant task for both types of action-coding striatal neuron, and show they co-operate to promote action selection in learning and compete to promote action suppression in extinction. Separately, we derive a complete model of dopamine and spike-timing dependent cortico-striatal plasticity from in vitro data. We then show this model produces the predicted activity changes necessary for learning and extinction in an operant task, a remarkable convergence of a bottom-up data-driven plasticity model with the top-down behavioural requirements of learning theory. Moreover, we show the complex dependencies of cortico-striatal plasticity are not only sufficient but necessary for learning and extinction. Validating the model, we show it can account for behavioural data describing extinction, renewal, and reacquisition, and replicate in vitro experimental data on cortico-striatal plasticity. By bridging the levels between the single synapse and behaviour, our model shows how striatum acts as the action-reinforcement interface.

No MeSH data available.


Related in: MedlinePlus

Expected overall weight change as a function of dopamine concentration.Here we plot the mean (line) and range (shading) of the overall weight change (sum of the plasticity factors ) at a given dopamine level , across every set of plasticity coefficients found by the search (Figure 10). (A) Separate plots for the found sets of D1 and D2 MSN coefficients, showing the dopamine dependence of each neuron type. (B) The sum of the individual MSN-type contributions in (A).
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4285402&req=5

pbio-1002034-g013: Expected overall weight change as a function of dopamine concentration.Here we plot the mean (line) and range (shading) of the overall weight change (sum of the plasticity factors ) at a given dopamine level , across every set of plasticity coefficients found by the search (Figure 10). (A) Separate plots for the found sets of D1 and D2 MSN coefficients, showing the dopamine dependence of each neuron type. (B) The sum of the individual MSN-type contributions in (A).

Mentions: In Figure 13 we plot the sum of the STDP kernel amplitudes as a function of dopamine concentration, which approximates the expected overall weight change for random trains of input and output spikes, for every successful coefficient set from the exhaustive search. The range of weight changes shown are hence consistent with successful action selection and suppression of the key action. We see that, if we plot the equivalent curve to that in [15] by not distinguishing D1 and D2 MSNs, then our model predicts that the average total measured weight change approximates the curve in [15]. However, the range of total weight change we observed, consistent with successful selection of the key action, covers both LTD and LTP at many dopamine levels. This is accounted for in the model by its prediction that increasing dopamine switches D1 MSN synapses from LTD to LTP and D2 MSN synapses from LTP to LTD. Our results thus suggest that the dependence on both dopamine receptor and dopamine concentration forms the minimal model of cortico-striatal plasticity.


A new framework for cortico-striatal plasticity: behavioural theory meets in vitro data at the reinforcement-action interface.

Gurney KN, Humphries MD, Redgrave P - PLoS Biol. (2015)

Expected overall weight change as a function of dopamine concentration.Here we plot the mean (line) and range (shading) of the overall weight change (sum of the plasticity factors ) at a given dopamine level , across every set of plasticity coefficients found by the search (Figure 10). (A) Separate plots for the found sets of D1 and D2 MSN coefficients, showing the dopamine dependence of each neuron type. (B) The sum of the individual MSN-type contributions in (A).
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4285402&req=5

pbio-1002034-g013: Expected overall weight change as a function of dopamine concentration.Here we plot the mean (line) and range (shading) of the overall weight change (sum of the plasticity factors ) at a given dopamine level , across every set of plasticity coefficients found by the search (Figure 10). (A) Separate plots for the found sets of D1 and D2 MSN coefficients, showing the dopamine dependence of each neuron type. (B) The sum of the individual MSN-type contributions in (A).
Mentions: In Figure 13 we plot the sum of the STDP kernel amplitudes as a function of dopamine concentration, which approximates the expected overall weight change for random trains of input and output spikes, for every successful coefficient set from the exhaustive search. The range of weight changes shown are hence consistent with successful action selection and suppression of the key action. We see that, if we plot the equivalent curve to that in [15] by not distinguishing D1 and D2 MSNs, then our model predicts that the average total measured weight change approximates the curve in [15]. However, the range of total weight change we observed, consistent with successful selection of the key action, covers both LTD and LTP at many dopamine levels. This is accounted for in the model by its prediction that increasing dopamine switches D1 MSN synapses from LTD to LTP and D2 MSN synapses from LTP to LTD. Our results thus suggest that the dependence on both dopamine receptor and dopamine concentration forms the minimal model of cortico-striatal plasticity.

Bottom Line: Third, the two types of striatal output neuron have apparently opposite effects on action selection.Validating the model, we show it can account for behavioural data describing extinction, renewal, and reacquisition, and replicate in vitro experimental data on cortico-striatal plasticity.By bridging the levels between the single synapse and behaviour, our model shows how striatum acts as the action-reinforcement interface.

View Article: PubMed Central - PubMed

Affiliation: Department of Psychology, Adaptive Behaviour Research Group, University of Sheffield, United Kingdom; INSIGNEO Institute for In Silico Medicine, University of Sheffield, United Kingdom.

ABSTRACT
Operant learning requires that reinforcement signals interact with action representations at a suitable neural interface. Much evidence suggests that this occurs when phasic dopamine, acting as a reinforcement prediction error, gates plasticity at cortico-striatal synapses, and thereby changes the future likelihood of selecting the action(s) coded by striatal neurons. But this hypothesis faces serious challenges. First, cortico-striatal plasticity is inexplicably complex, depending on spike timing, dopamine level, and dopamine receptor type. Second, there is a credit assignment problem-action selection signals occur long before the consequent dopamine reinforcement signal. Third, the two types of striatal output neuron have apparently opposite effects on action selection. Whether these factors rule out the interface hypothesis and how they interact to produce reinforcement learning is unknown. We present a computational framework that addresses these challenges. We first predict the expected activity changes over an operant task for both types of action-coding striatal neuron, and show they co-operate to promote action selection in learning and compete to promote action suppression in extinction. Separately, we derive a complete model of dopamine and spike-timing dependent cortico-striatal plasticity from in vitro data. We then show this model produces the predicted activity changes necessary for learning and extinction in an operant task, a remarkable convergence of a bottom-up data-driven plasticity model with the top-down behavioural requirements of learning theory. Moreover, we show the complex dependencies of cortico-striatal plasticity are not only sufficient but necessary for learning and extinction. Validating the model, we show it can account for behavioural data describing extinction, renewal, and reacquisition, and replicate in vitro experimental data on cortico-striatal plasticity. By bridging the levels between the single synapse and behaviour, our model shows how striatum acts as the action-reinforcement interface.

No MeSH data available.


Related in: MedlinePlus