Limits...
Short-term temporal discounting of reward value in human ventral striatum.

Gregorios-Pippas L, Tobler PN, Schultz W - J. Neurophysiol. (2009)

Bottom Line: We demonstrated hyperbolic and exponential decreases of striatal responses to reward predicting stimuli within this time range, irrespective of changes in reward rate.These data suggest that delays of a few seconds affect the neural processing of predicted reward value in the ventral striatum and engage the temporal sensitivity of reward responses.Comparisons with electrophysiological animal data suggest that ventral striatal reward discounting may involve dopaminergic and orbitofrontal inputs.

View Article: PubMed Central - PubMed

Affiliation: Department of Physiology, Development and Neuroscience, University of Cambridge, Cambridge, UK.

ABSTRACT
Delayed rewards lose their value for economic decisions and constitute weaker reinforcers for learning. Temporal discounting of reward value already occurs within a few seconds in animals, which allows investigations of the underlying neurophysiological mechanisms. However, it is difficult to relate these mechanisms to human discounting behavior, which is usually studied over days and months and may engage different brain processes. Our study aimed to bridge the gap by using very short delays and measuring human functional magnetic resonance responses in one of the key reward centers of the brain, the ventral striatum. We used psychometric methods to assess subjective timing and valuation of monetary rewards with delays of 4.0-13.5 s. We demonstrated hyperbolic and exponential decreases of striatal responses to reward predicting stimuli within this time range, irrespective of changes in reward rate. Lower reward magnitudes induced steeper behavioral and striatal discounting. By contrast, striatal responses following the delivery of reward reflected the uncertainty in subjective timing associated with delayed rewards rather than value discounting. These data suggest that delays of a few seconds affect the neural processing of predicted reward value in the ventral striatum and engage the temporal sensitivity of reward responses. Comparisons with electrophysiological animal data suggest that ventral striatal reward discounting may involve dopaminergic and orbitofrontal inputs.

Show MeSH

Related in: MedlinePlus

Decreases of blood-oxygen-level-dependent (BOLD) responses to reward predicting stimuli in ventral striatum during temporal discounting. A: anatomical position of BOLD responses revealed by contrast estimates based on regression using the general linear model (random effects group analysis). Circle shows cluster with significant activation at P < 0.05 (ventral striatum small volume corrected). Peak activation occurred at voxel at −18/14/-8. BOLD responses were regressed on the individual indifference values measured in each participant in the intertemporal choice task, averaged across fixed and adjusted ITI schedules for delays of 4, 6, 9, and 13.5 s. Contrast estimates were linear combinations of slope regression coefficient estimates, beta. B: 2 control analyses supporting decreases of BOLD responses in ventral striatum during temporal discounting. Yellow (upper) circle: better correlation with discounted than undiscounted outcomes (P < 0.07, ventral striatum small volume correction). The regression used the differences: individually measured (discounted) indifference values minus constant (undiscounted) 20 £ outcomes instead of indifference values alone. Blue (lower) circle (same data set): slightly better correlation with individual than population averaged indifference values (P < 0.05 uncorrected). The regression used the differences: individual indifference values minus average. C: time courses of BOLD responses to stimuli predicting different reward delays, measured at peak voxel of circled area shown in A. Response peaks (green rectangles) decreased with increasing temporal delays in the 7 participants showing behavioral discounting (top) but not in the 7 nondiscounters (bottom) in fixed (left) and adjusted ITI schedules (right; peak voxels of region circled in A). Insets: PIP estimated subjective reward delays (means across 7 discounters and 7 nondiscounters, respectively). Time = 0 s refers to onset of delay predicting stimuli. D: decreases of BOLD responses to reward delay predicting stimuli. Data were averaged separately in participants showing strong or weak discounting (7 participants each, classified by median split of behavioral discounting). Percentages of signal change were measured at peaks of time courses (4 s after stimulus onset, shaded intervals in C) at peak voxel of BOLD response shown in A. The fitted curves were based on the mean subjective PIP task-estimated delays from 7 discounters and 7 nondiscounters, respectively, for objective intervals of 4, 6, 9, and 13.5 s and conformed to hyperbolic (top) and exponential functions (bottom). Data are means ± SE. All measures were from peak voxel of circled area shown in A.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC2666398&req=5

f3: Decreases of blood-oxygen-level-dependent (BOLD) responses to reward predicting stimuli in ventral striatum during temporal discounting. A: anatomical position of BOLD responses revealed by contrast estimates based on regression using the general linear model (random effects group analysis). Circle shows cluster with significant activation at P < 0.05 (ventral striatum small volume corrected). Peak activation occurred at voxel at −18/14/-8. BOLD responses were regressed on the individual indifference values measured in each participant in the intertemporal choice task, averaged across fixed and adjusted ITI schedules for delays of 4, 6, 9, and 13.5 s. Contrast estimates were linear combinations of slope regression coefficient estimates, beta. B: 2 control analyses supporting decreases of BOLD responses in ventral striatum during temporal discounting. Yellow (upper) circle: better correlation with discounted than undiscounted outcomes (P < 0.07, ventral striatum small volume correction). The regression used the differences: individually measured (discounted) indifference values minus constant (undiscounted) 20 £ outcomes instead of indifference values alone. Blue (lower) circle (same data set): slightly better correlation with individual than population averaged indifference values (P < 0.05 uncorrected). The regression used the differences: individual indifference values minus average. C: time courses of BOLD responses to stimuli predicting different reward delays, measured at peak voxel of circled area shown in A. Response peaks (green rectangles) decreased with increasing temporal delays in the 7 participants showing behavioral discounting (top) but not in the 7 nondiscounters (bottom) in fixed (left) and adjusted ITI schedules (right; peak voxels of region circled in A). Insets: PIP estimated subjective reward delays (means across 7 discounters and 7 nondiscounters, respectively). Time = 0 s refers to onset of delay predicting stimuli. D: decreases of BOLD responses to reward delay predicting stimuli. Data were averaged separately in participants showing strong or weak discounting (7 participants each, classified by median split of behavioral discounting). Percentages of signal change were measured at peaks of time courses (4 s after stimulus onset, shaded intervals in C) at peak voxel of BOLD response shown in A. The fitted curves were based on the mean subjective PIP task-estimated delays from 7 discounters and 7 nondiscounters, respectively, for objective intervals of 4, 6, 9, and 13.5 s and conformed to hyperbolic (top) and exponential functions (bottom). Data are means ± SE. All measures were from peak voxel of circled area shown in A.

Mentions: We regressed BOLD responses to the four stimuli against the behavioral indifference values of each participant measured in the intertemporal choice task, irrespective of any particular discounting model. The regression identified one large group and two small groups of voxels in the striatum in which BOLD responses decreased across the four delays according to the individual indifference values. Subsequent analysis revealed statistical significance with small volume correction in the large ventral striatal group of voxels (circle in Fig. 3A; P < 0.05). The activation was significant for both one common set of regressors for the four delays in the two ITI schedules (8 regressors; P < 0.05) and for two separate sets of regressors for the two ITI schedules (2 × 4 regressors; P < 0.05). These data suggest that predictive reward value signals in the human ventral striatum decreased substantially when rewards were delayed by a few seconds, closely paralleling the discounting of subjective reward value measured by pleasantness ratings and behavioral indifference values.


Short-term temporal discounting of reward value in human ventral striatum.

Gregorios-Pippas L, Tobler PN, Schultz W - J. Neurophysiol. (2009)

Decreases of blood-oxygen-level-dependent (BOLD) responses to reward predicting stimuli in ventral striatum during temporal discounting. A: anatomical position of BOLD responses revealed by contrast estimates based on regression using the general linear model (random effects group analysis). Circle shows cluster with significant activation at P < 0.05 (ventral striatum small volume corrected). Peak activation occurred at voxel at −18/14/-8. BOLD responses were regressed on the individual indifference values measured in each participant in the intertemporal choice task, averaged across fixed and adjusted ITI schedules for delays of 4, 6, 9, and 13.5 s. Contrast estimates were linear combinations of slope regression coefficient estimates, beta. B: 2 control analyses supporting decreases of BOLD responses in ventral striatum during temporal discounting. Yellow (upper) circle: better correlation with discounted than undiscounted outcomes (P < 0.07, ventral striatum small volume correction). The regression used the differences: individually measured (discounted) indifference values minus constant (undiscounted) 20 £ outcomes instead of indifference values alone. Blue (lower) circle (same data set): slightly better correlation with individual than population averaged indifference values (P < 0.05 uncorrected). The regression used the differences: individual indifference values minus average. C: time courses of BOLD responses to stimuli predicting different reward delays, measured at peak voxel of circled area shown in A. Response peaks (green rectangles) decreased with increasing temporal delays in the 7 participants showing behavioral discounting (top) but not in the 7 nondiscounters (bottom) in fixed (left) and adjusted ITI schedules (right; peak voxels of region circled in A). Insets: PIP estimated subjective reward delays (means across 7 discounters and 7 nondiscounters, respectively). Time = 0 s refers to onset of delay predicting stimuli. D: decreases of BOLD responses to reward delay predicting stimuli. Data were averaged separately in participants showing strong or weak discounting (7 participants each, classified by median split of behavioral discounting). Percentages of signal change were measured at peaks of time courses (4 s after stimulus onset, shaded intervals in C) at peak voxel of BOLD response shown in A. The fitted curves were based on the mean subjective PIP task-estimated delays from 7 discounters and 7 nondiscounters, respectively, for objective intervals of 4, 6, 9, and 13.5 s and conformed to hyperbolic (top) and exponential functions (bottom). Data are means ± SE. All measures were from peak voxel of circled area shown in A.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC2666398&req=5

f3: Decreases of blood-oxygen-level-dependent (BOLD) responses to reward predicting stimuli in ventral striatum during temporal discounting. A: anatomical position of BOLD responses revealed by contrast estimates based on regression using the general linear model (random effects group analysis). Circle shows cluster with significant activation at P < 0.05 (ventral striatum small volume corrected). Peak activation occurred at voxel at −18/14/-8. BOLD responses were regressed on the individual indifference values measured in each participant in the intertemporal choice task, averaged across fixed and adjusted ITI schedules for delays of 4, 6, 9, and 13.5 s. Contrast estimates were linear combinations of slope regression coefficient estimates, beta. B: 2 control analyses supporting decreases of BOLD responses in ventral striatum during temporal discounting. Yellow (upper) circle: better correlation with discounted than undiscounted outcomes (P < 0.07, ventral striatum small volume correction). The regression used the differences: individually measured (discounted) indifference values minus constant (undiscounted) 20 £ outcomes instead of indifference values alone. Blue (lower) circle (same data set): slightly better correlation with individual than population averaged indifference values (P < 0.05 uncorrected). The regression used the differences: individual indifference values minus average. C: time courses of BOLD responses to stimuli predicting different reward delays, measured at peak voxel of circled area shown in A. Response peaks (green rectangles) decreased with increasing temporal delays in the 7 participants showing behavioral discounting (top) but not in the 7 nondiscounters (bottom) in fixed (left) and adjusted ITI schedules (right; peak voxels of region circled in A). Insets: PIP estimated subjective reward delays (means across 7 discounters and 7 nondiscounters, respectively). Time = 0 s refers to onset of delay predicting stimuli. D: decreases of BOLD responses to reward delay predicting stimuli. Data were averaged separately in participants showing strong or weak discounting (7 participants each, classified by median split of behavioral discounting). Percentages of signal change were measured at peaks of time courses (4 s after stimulus onset, shaded intervals in C) at peak voxel of BOLD response shown in A. The fitted curves were based on the mean subjective PIP task-estimated delays from 7 discounters and 7 nondiscounters, respectively, for objective intervals of 4, 6, 9, and 13.5 s and conformed to hyperbolic (top) and exponential functions (bottom). Data are means ± SE. All measures were from peak voxel of circled area shown in A.
Mentions: We regressed BOLD responses to the four stimuli against the behavioral indifference values of each participant measured in the intertemporal choice task, irrespective of any particular discounting model. The regression identified one large group and two small groups of voxels in the striatum in which BOLD responses decreased across the four delays according to the individual indifference values. Subsequent analysis revealed statistical significance with small volume correction in the large ventral striatal group of voxels (circle in Fig. 3A; P < 0.05). The activation was significant for both one common set of regressors for the four delays in the two ITI schedules (8 regressors; P < 0.05) and for two separate sets of regressors for the two ITI schedules (2 × 4 regressors; P < 0.05). These data suggest that predictive reward value signals in the human ventral striatum decreased substantially when rewards were delayed by a few seconds, closely paralleling the discounting of subjective reward value measured by pleasantness ratings and behavioral indifference values.

Bottom Line: We demonstrated hyperbolic and exponential decreases of striatal responses to reward predicting stimuli within this time range, irrespective of changes in reward rate.These data suggest that delays of a few seconds affect the neural processing of predicted reward value in the ventral striatum and engage the temporal sensitivity of reward responses.Comparisons with electrophysiological animal data suggest that ventral striatal reward discounting may involve dopaminergic and orbitofrontal inputs.

View Article: PubMed Central - PubMed

Affiliation: Department of Physiology, Development and Neuroscience, University of Cambridge, Cambridge, UK.

ABSTRACT
Delayed rewards lose their value for economic decisions and constitute weaker reinforcers for learning. Temporal discounting of reward value already occurs within a few seconds in animals, which allows investigations of the underlying neurophysiological mechanisms. However, it is difficult to relate these mechanisms to human discounting behavior, which is usually studied over days and months and may engage different brain processes. Our study aimed to bridge the gap by using very short delays and measuring human functional magnetic resonance responses in one of the key reward centers of the brain, the ventral striatum. We used psychometric methods to assess subjective timing and valuation of monetary rewards with delays of 4.0-13.5 s. We demonstrated hyperbolic and exponential decreases of striatal responses to reward predicting stimuli within this time range, irrespective of changes in reward rate. Lower reward magnitudes induced steeper behavioral and striatal discounting. By contrast, striatal responses following the delivery of reward reflected the uncertainty in subjective timing associated with delayed rewards rather than value discounting. These data suggest that delays of a few seconds affect the neural processing of predicted reward value in the ventral striatum and engage the temporal sensitivity of reward responses. Comparisons with electrophysiological animal data suggest that ventral striatal reward discounting may involve dopaminergic and orbitofrontal inputs.

Show MeSH
Related in: MedlinePlus