Limits...
Tonic dopamine modulates exploitation of reward learning.

Beeler JA, Daw N, Frazier CR, Zhuang X - Front Behav Neurosci (2010)

Bottom Line: In this "closed economy" paradigm, subjects earn all of their food by pressing either of two levers, but the relative cost for food on each lever shifts frequently.We fit the lever choice data using reinforcement learning models to assess the distinction between acquisition and expression the models formalize.These data suggest that dopamine modulates the degree to which prior learning biases action selection and consequently alters the expression of learned, motivated behavior.

View Article: PubMed Central - PubMed

Affiliation: Department of Neurobiology, University of Chicago, Chicago, IL, USA. jabeeler@uchicago.edu

ABSTRACT
The impact of dopamine on adaptive behavior in a naturalistic environment is largely unexamined. Experimental work suggests that phasic dopamine is central to reinforcement learning whereas tonic dopamine may modulate performance without altering learning per se; however, this idea has not been developed formally or integrated with computational models of dopamine function. We quantitatively evaluate the role of tonic dopamine in these functions by studying the behavior of hyperdopaminergic DAT knockdown mice in an instrumental task in a semi-naturalistic homecage environment. In this "closed economy" paradigm, subjects earn all of their food by pressing either of two levers, but the relative cost for food on each lever shifts frequently. Compared to wild-type mice, hyperdopaminergic mice allocate more lever presses on high-cost levers, thus working harder to earn a given amount of food and maintain their body weight. However, both groups show a similarly quick reaction to shifts in lever cost, suggesting that the hyperdominergic mice are not slower at detecting changes, as with a learning deficit. We fit the lever choice data using reinforcement learning models to assess the distinction between acquisition and expression the models formalize. In these analyses, hyperdopaminergic mice displayed normal learning from recent reward history but diminished capacity to exploit this learning: a reduced coupling between choice and reward history. These data suggest that dopamine modulates the degree to which prior learning biases action selection and consequently alters the expression of learned, motivated behavior.

No MeSH data available.


Effort and earned rewards when the price of the high and low cost levers does not switch. Average lever presses on the (A) low cost and (B) high cost lever and average pellets earned on (C) low and (D) high cost levers as the price of the high cost lever increases across days. No significant genotype differences across panels. Error bars = S.E.M, N = 6.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC2991243&req=5

Figure 4: Effort and earned rewards when the price of the high and low cost levers does not switch. Average lever presses on the (A) low cost and (B) high cost lever and average pellets earned on (C) low and (D) high cost levers as the price of the high cost lever increases across days. No significant genotype differences across panels. Error bars = S.E.M, N = 6.

Mentions: There are several potential explanations to the behavioral results described. The DATkd mice may be insensitive to costs and/or might derive some intrinsic value from lever pressing itself. To test these, we conducted a similar experiment with a cheap and expensive lever but which lever was cheap and expensive remained constant. We observe no significant differences between the groups in the stationary version of the paradigm (Figures 4A–D). This clearly indicates that the DATkd do not derive an intrinsic value from lever pressing. More importantly, though the results in the switching paradigm are consistent with a reduced sensitivity to cost in the DATkd, this experiment indicates that they are not indifferent to cost. Thus, their apparent reduced sensitivity to cost in the switching paradigm arises as a consequence of how they use reward (and cost) history to determine their behavioral strategy in a dynamic environment and not as a result of generalized indifference to cost.


Tonic dopamine modulates exploitation of reward learning.

Beeler JA, Daw N, Frazier CR, Zhuang X - Front Behav Neurosci (2010)

Effort and earned rewards when the price of the high and low cost levers does not switch. Average lever presses on the (A) low cost and (B) high cost lever and average pellets earned on (C) low and (D) high cost levers as the price of the high cost lever increases across days. No significant genotype differences across panels. Error bars = S.E.M, N = 6.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC2991243&req=5

Figure 4: Effort and earned rewards when the price of the high and low cost levers does not switch. Average lever presses on the (A) low cost and (B) high cost lever and average pellets earned on (C) low and (D) high cost levers as the price of the high cost lever increases across days. No significant genotype differences across panels. Error bars = S.E.M, N = 6.
Mentions: There are several potential explanations to the behavioral results described. The DATkd mice may be insensitive to costs and/or might derive some intrinsic value from lever pressing itself. To test these, we conducted a similar experiment with a cheap and expensive lever but which lever was cheap and expensive remained constant. We observe no significant differences between the groups in the stationary version of the paradigm (Figures 4A–D). This clearly indicates that the DATkd do not derive an intrinsic value from lever pressing. More importantly, though the results in the switching paradigm are consistent with a reduced sensitivity to cost in the DATkd, this experiment indicates that they are not indifferent to cost. Thus, their apparent reduced sensitivity to cost in the switching paradigm arises as a consequence of how they use reward (and cost) history to determine their behavioral strategy in a dynamic environment and not as a result of generalized indifference to cost.

Bottom Line: In this "closed economy" paradigm, subjects earn all of their food by pressing either of two levers, but the relative cost for food on each lever shifts frequently.We fit the lever choice data using reinforcement learning models to assess the distinction between acquisition and expression the models formalize.These data suggest that dopamine modulates the degree to which prior learning biases action selection and consequently alters the expression of learned, motivated behavior.

View Article: PubMed Central - PubMed

Affiliation: Department of Neurobiology, University of Chicago, Chicago, IL, USA. jabeeler@uchicago.edu

ABSTRACT
The impact of dopamine on adaptive behavior in a naturalistic environment is largely unexamined. Experimental work suggests that phasic dopamine is central to reinforcement learning whereas tonic dopamine may modulate performance without altering learning per se; however, this idea has not been developed formally or integrated with computational models of dopamine function. We quantitatively evaluate the role of tonic dopamine in these functions by studying the behavior of hyperdopaminergic DAT knockdown mice in an instrumental task in a semi-naturalistic homecage environment. In this "closed economy" paradigm, subjects earn all of their food by pressing either of two levers, but the relative cost for food on each lever shifts frequently. Compared to wild-type mice, hyperdopaminergic mice allocate more lever presses on high-cost levers, thus working harder to earn a given amount of food and maintain their body weight. However, both groups show a similarly quick reaction to shifts in lever cost, suggesting that the hyperdominergic mice are not slower at detecting changes, as with a learning deficit. We fit the lever choice data using reinforcement learning models to assess the distinction between acquisition and expression the models formalize. In these analyses, hyperdopaminergic mice displayed normal learning from recent reward history but diminished capacity to exploit this learning: a reduced coupling between choice and reward history. These data suggest that dopamine modulates the degree to which prior learning biases action selection and consequently alters the expression of learned, motivated behavior.

No MeSH data available.