Limits...
Dopamine signals for reward value and risk: basic and recent data.

Schultz W - Behav Brain Funct (2010)

Bottom Line: The prediction error response occurs only with reward; it is scaled by the risk of predicted reward.Neurophysiological studies reveal phasic dopamine signals that transmit information related predominantly but not exclusively to reward.Although not being entirely homogeneous, the dopamine signal is more restricted and stereotyped than neuronal activity in most other brain structures involved in goal directed behavior.

View Article: PubMed Central - HTML - PubMed

Affiliation: Department of Physiology, Development and Neuroscience, University of Cambridge, Downing Street, Cambridge, UK. ws234@cam.ac.uk

ABSTRACT

Background: Previous lesion, electrical self-stimulation and drug addiction studies suggest that the midbrain dopamine systems are parts of the reward system of the brain. This review provides an updated overview about the basic signals of dopamine neurons to environmental stimuli.

Methods: The described experiments used standard behavioral and neurophysiological methods to record the activity of single dopamine neurons in awake monkeys during specific behavioral tasks.

Results: Dopamine neurons show phasic activations to external stimuli. The signal reflects reward, physical salience, risk and punishment, in descending order of fractions of responding neurons. Expected reward value is a key decision variable for economic choices. The reward response codes reward value, probability and their summed product, expected value. The neurons code reward value as it differs from prediction, thus fulfilling the basic requirement for a bidirectional prediction error teaching signal postulated by learning theory. This response is scaled in units of standard deviation. By contrast, relatively few dopamine neurons show the phasic activation following punishers and conditioned aversive stimuli, suggesting a lack of relationship of the reward response to general attention and arousal. Large proportions of dopamine neurons are also activated by intense, physically salient stimuli. This response is enhanced when the stimuli are novel; it appears to be distinct from the reward value signal. Dopamine neurons show also unspecific activations to non-rewarding stimuli that are possibly due to generalization by similar stimuli and pseudoconditioning by primary rewards. These activations are shorter than reward responses and are often followed by depression of activity. A separate, slower dopamine signal informs about risk, another important decision variable. The prediction error response occurs only with reward; it is scaled by the risk of predicted reward.

Conclusions: Neurophysiological studies reveal phasic dopamine signals that transmit information related predominantly but not exclusively to reward. Although not being entirely homogeneous, the dopamine signal is more restricted and stereotyped than neuronal activity in most other brain structures involved in goal directed behavior.

Show MeSH

Related in: MedlinePlus

Phasic activations of neurophysiological impulse activity of dopamine neurons. A: Phasic activations following primary rewards. B: Phasic activations following conditioned, reward predicting stimuli. C: Top: Lack of phasic activation following primary aversive air puff. Bottom: substantial activating population response following conditioned aversive stimuli when stimulus generalization by appetitive stimuli is not ruled out; grey: population response to conditioned visual aversive stimulus when appetitive stimulus is also visual; black: lack of population response to conditioned visual aversive stimulus when appetitive stimulus is auditory. D: Phasic activations following physically intense stimuli. These activations are modulated by the novelty of the stimuli but do not occur to novelty per se. E: Left: Shorter and smaller activations followed frequently by depressions induced by unrewarded control stimuli (black) compared to responses following reward predicting stimuli (grey). Right: Activations to delay predicting stimuli show initial, poorly graded activation component (left of line) and subsequent, graded value component inversely reflecting increasing delays (curves from top to bottom). Time scale (500 ms) applies to all panels A-E. Data from previous work [29,31-33,43,59].
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC2876988&req=5

Figure 1: Phasic activations of neurophysiological impulse activity of dopamine neurons. A: Phasic activations following primary rewards. B: Phasic activations following conditioned, reward predicting stimuli. C: Top: Lack of phasic activation following primary aversive air puff. Bottom: substantial activating population response following conditioned aversive stimuli when stimulus generalization by appetitive stimuli is not ruled out; grey: population response to conditioned visual aversive stimulus when appetitive stimulus is also visual; black: lack of population response to conditioned visual aversive stimulus when appetitive stimulus is auditory. D: Phasic activations following physically intense stimuli. These activations are modulated by the novelty of the stimuli but do not occur to novelty per se. E: Left: Shorter and smaller activations followed frequently by depressions induced by unrewarded control stimuli (black) compared to responses following reward predicting stimuli (grey). Right: Activations to delay predicting stimuli show initial, poorly graded activation component (left of line) and subsequent, graded value component inversely reflecting increasing delays (curves from top to bottom). Time scale (500 ms) applies to all panels A-E. Data from previous work [29,31-33,43,59].

Mentions: The majority of midbrain dopamine neurons (75-80%) show rather stereotyped, phasic activations with latencies of <100 ms and durations of <200 ms following temporally unpredicted food and liquid rewards (Figure 1A). This burst response depends on the activation and plasticity of glutamatergic NMDA and AMPA receptors located on dopamine neurons [8-12]. The burst is critical for behavioral learning of appetitive tasks such as conditioned place preference and T-maze choices for food or cocaine reward and for conditioned fear responses [9].


Dopamine signals for reward value and risk: basic and recent data.

Schultz W - Behav Brain Funct (2010)

Phasic activations of neurophysiological impulse activity of dopamine neurons. A: Phasic activations following primary rewards. B: Phasic activations following conditioned, reward predicting stimuli. C: Top: Lack of phasic activation following primary aversive air puff. Bottom: substantial activating population response following conditioned aversive stimuli when stimulus generalization by appetitive stimuli is not ruled out; grey: population response to conditioned visual aversive stimulus when appetitive stimulus is also visual; black: lack of population response to conditioned visual aversive stimulus when appetitive stimulus is auditory. D: Phasic activations following physically intense stimuli. These activations are modulated by the novelty of the stimuli but do not occur to novelty per se. E: Left: Shorter and smaller activations followed frequently by depressions induced by unrewarded control stimuli (black) compared to responses following reward predicting stimuli (grey). Right: Activations to delay predicting stimuli show initial, poorly graded activation component (left of line) and subsequent, graded value component inversely reflecting increasing delays (curves from top to bottom). Time scale (500 ms) applies to all panels A-E. Data from previous work [29,31-33,43,59].
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC2876988&req=5

Figure 1: Phasic activations of neurophysiological impulse activity of dopamine neurons. A: Phasic activations following primary rewards. B: Phasic activations following conditioned, reward predicting stimuli. C: Top: Lack of phasic activation following primary aversive air puff. Bottom: substantial activating population response following conditioned aversive stimuli when stimulus generalization by appetitive stimuli is not ruled out; grey: population response to conditioned visual aversive stimulus when appetitive stimulus is also visual; black: lack of population response to conditioned visual aversive stimulus when appetitive stimulus is auditory. D: Phasic activations following physically intense stimuli. These activations are modulated by the novelty of the stimuli but do not occur to novelty per se. E: Left: Shorter and smaller activations followed frequently by depressions induced by unrewarded control stimuli (black) compared to responses following reward predicting stimuli (grey). Right: Activations to delay predicting stimuli show initial, poorly graded activation component (left of line) and subsequent, graded value component inversely reflecting increasing delays (curves from top to bottom). Time scale (500 ms) applies to all panels A-E. Data from previous work [29,31-33,43,59].
Mentions: The majority of midbrain dopamine neurons (75-80%) show rather stereotyped, phasic activations with latencies of <100 ms and durations of <200 ms following temporally unpredicted food and liquid rewards (Figure 1A). This burst response depends on the activation and plasticity of glutamatergic NMDA and AMPA receptors located on dopamine neurons [8-12]. The burst is critical for behavioral learning of appetitive tasks such as conditioned place preference and T-maze choices for food or cocaine reward and for conditioned fear responses [9].

Bottom Line: The prediction error response occurs only with reward; it is scaled by the risk of predicted reward.Neurophysiological studies reveal phasic dopamine signals that transmit information related predominantly but not exclusively to reward.Although not being entirely homogeneous, the dopamine signal is more restricted and stereotyped than neuronal activity in most other brain structures involved in goal directed behavior.

View Article: PubMed Central - HTML - PubMed

Affiliation: Department of Physiology, Development and Neuroscience, University of Cambridge, Downing Street, Cambridge, UK. ws234@cam.ac.uk

ABSTRACT

Background: Previous lesion, electrical self-stimulation and drug addiction studies suggest that the midbrain dopamine systems are parts of the reward system of the brain. This review provides an updated overview about the basic signals of dopamine neurons to environmental stimuli.

Methods: The described experiments used standard behavioral and neurophysiological methods to record the activity of single dopamine neurons in awake monkeys during specific behavioral tasks.

Results: Dopamine neurons show phasic activations to external stimuli. The signal reflects reward, physical salience, risk and punishment, in descending order of fractions of responding neurons. Expected reward value is a key decision variable for economic choices. The reward response codes reward value, probability and their summed product, expected value. The neurons code reward value as it differs from prediction, thus fulfilling the basic requirement for a bidirectional prediction error teaching signal postulated by learning theory. This response is scaled in units of standard deviation. By contrast, relatively few dopamine neurons show the phasic activation following punishers and conditioned aversive stimuli, suggesting a lack of relationship of the reward response to general attention and arousal. Large proportions of dopamine neurons are also activated by intense, physically salient stimuli. This response is enhanced when the stimuli are novel; it appears to be distinct from the reward value signal. Dopamine neurons show also unspecific activations to non-rewarding stimuli that are possibly due to generalization by similar stimuli and pseudoconditioning by primary rewards. These activations are shorter than reward responses and are often followed by depression of activity. A separate, slower dopamine signal informs about risk, another important decision variable. The prediction error response occurs only with reward; it is scaled by the risk of predicted reward.

Conclusions: Neurophysiological studies reveal phasic dopamine signals that transmit information related predominantly but not exclusively to reward. Although not being entirely homogeneous, the dopamine signal is more restricted and stereotyped than neuronal activity in most other brain structures involved in goal directed behavior.

Show MeSH
Related in: MedlinePlus