Limits...
Planning activity for internally generated reward goals in monkey amygdala neurons.

Hernádi I, Grabenhorst F, Schultz W - Nat. Neurosci. (2015)

Bottom Line: The best rewards are often distant and can only be achieved by planning and decision-making over several steps.Such prospective activity could underlie the formation and pursuit of internal plans characteristic of goal-directed behavior.The existence of neuronal planning activity in the amygdala suggests that this structure is important in guiding behavior toward internally generated, distant goals.

View Article: PubMed Central - PubMed

Affiliation: Department of Physiology, Development and Neuroscience, University of Cambridge, Cambridge, UK.

ABSTRACT
The best rewards are often distant and can only be achieved by planning and decision-making over several steps. We designed a multi-step choice task in which monkeys followed internal plans to save rewards toward self-defined goals. During this self-controlled behavior, amygdala neurons showed future-oriented activity that reflected the animal's plan to obtain specific rewards several trials ahead. This prospective activity encoded crucial components of the animal's plan, including value and length of the planned choice sequence. It began on initial trials when a plan would be formed, reappeared step by step until reward receipt, and readily updated with a new sequence. It predicted performance, including errors, and typically disappeared during instructed behavior. Such prospective activity could underlie the formation and pursuit of internal plans characteristic of goal-directed behavior. The existence of neuronal planning activity in the amygdala suggests that this structure is important in guiding behavior toward internally generated, distant goals.

Show MeSH

Related in: MedlinePlus

Adaptation dynamics of planning activity, reward proximity control. (a) Sequence-by-sequence adaptation in a single neuron encoding sequence length. Activity changes from spend to save trials (dashed lines) reflected changes in sequence length between successive sequences. Gray curves: sequence-averaged activity (thick line) and trial-by-trial activity (thin line). Green curve: sequence length. Blue curve: within-sequence reward proximity. Arrows: examples for activity changes scaling with sequence length changes. Colored boxes indicate sequences and corresponding lengths. (b) Linear regression of activity of the neuron in (a) on sequence length (left, n = 41), difference in length between subsequent sequences (ΔSequence length, middle, n = 7), and reward proximity (right, n = 41). (c) Population data. Left: sequence value responses (n = 61); activity changes at sequence transitions reflected changes in sequence value (linear regression). Middle: sequence length responses (n = 55); activity changes reflected changes in sequence length. Right: Population activity (sequence value and sequence length responses, n = 116) was unrelated to within-sequence reward proximity. (d) Regression betas for planning activity and reward proximity (n = 116 sequence value and sequence length responses, Kolmogorov-Smirnov test). (e) Behavioral-neuronal adaptation in sequence value neurons. Upper: With a new testing session, planning activity adapted readily to current interest rate, in-step with behavior (r = 0.82, P = 1.7 × 10−4; both Medians = 1, n = 61). Lower: Neurons typically reached adaptation criterion within the first sequence (Median = −3, implying adaption within 3 trials before end of first sequence, t60 = −10.17, P = 1.0 × 10−14, one-sample t-test).
© Copyright Policy
Related In: Results  -  Collection


getmorefigures.php?uid=PMC4340753&req=5

Figure 5: Adaptation dynamics of planning activity, reward proximity control. (a) Sequence-by-sequence adaptation in a single neuron encoding sequence length. Activity changes from spend to save trials (dashed lines) reflected changes in sequence length between successive sequences. Gray curves: sequence-averaged activity (thick line) and trial-by-trial activity (thin line). Green curve: sequence length. Blue curve: within-sequence reward proximity. Arrows: examples for activity changes scaling with sequence length changes. Colored boxes indicate sequences and corresponding lengths. (b) Linear regression of activity of the neuron in (a) on sequence length (left, n = 41), difference in length between subsequent sequences (ΔSequence length, middle, n = 7), and reward proximity (right, n = 41). (c) Population data. Left: sequence value responses (n = 61); activity changes at sequence transitions reflected changes in sequence value (linear regression). Middle: sequence length responses (n = 55); activity changes reflected changes in sequence length. Right: Population activity (sequence value and sequence length responses, n = 116) was unrelated to within-sequence reward proximity. (d) Regression betas for planning activity and reward proximity (n = 116 sequence value and sequence length responses, Kolmogorov-Smirnov test). (e) Behavioral-neuronal adaptation in sequence value neurons. Upper: With a new testing session, planning activity adapted readily to current interest rate, in-step with behavior (r = 0.82, P = 1.7 × 10−4; both Medians = 1, n = 61). Lower: Neurons typically reached adaptation criterion within the first sequence (Median = −3, implying adaption within 3 trials before end of first sequence, t60 = −10.17, P = 1.0 × 10−14, one-sample t-test).

Mentions: If a neuron encoded components of the animals’ saving plan, its activity should update once a sequence is completed and begin to reflect properties of the subsequent sequence. Accordingly, we examined sequence transitions by comparing activity on spend trials and subsequent save trials (the last and first trials of two successive sequences). Figure 5a illustrates such transitions in a single neuron with planning activity related to sequence length. Transitions (dashed vertical lines) were marked by activity changes that scaled with changes in planned sequence length (compare thick gray and green lines). The neuron’s activity reflected planned sequence length within sequences (Fig. 5b left) and changes in planned sequence lengths at transitions (Fig. 5b middle). Activity was unrelated to within-sequence reward proximity (trials until reward, Fig. 5b right). Sequence-by-sequence adaptation was also evident in population activity (Fig. 5c, left and middle). Thus, planning activity adapted sequence-by-sequence to reflect changes in the animals’ internal plan.


Planning activity for internally generated reward goals in monkey amygdala neurons.

Hernádi I, Grabenhorst F, Schultz W - Nat. Neurosci. (2015)

Adaptation dynamics of planning activity, reward proximity control. (a) Sequence-by-sequence adaptation in a single neuron encoding sequence length. Activity changes from spend to save trials (dashed lines) reflected changes in sequence length between successive sequences. Gray curves: sequence-averaged activity (thick line) and trial-by-trial activity (thin line). Green curve: sequence length. Blue curve: within-sequence reward proximity. Arrows: examples for activity changes scaling with sequence length changes. Colored boxes indicate sequences and corresponding lengths. (b) Linear regression of activity of the neuron in (a) on sequence length (left, n = 41), difference in length between subsequent sequences (ΔSequence length, middle, n = 7), and reward proximity (right, n = 41). (c) Population data. Left: sequence value responses (n = 61); activity changes at sequence transitions reflected changes in sequence value (linear regression). Middle: sequence length responses (n = 55); activity changes reflected changes in sequence length. Right: Population activity (sequence value and sequence length responses, n = 116) was unrelated to within-sequence reward proximity. (d) Regression betas for planning activity and reward proximity (n = 116 sequence value and sequence length responses, Kolmogorov-Smirnov test). (e) Behavioral-neuronal adaptation in sequence value neurons. Upper: With a new testing session, planning activity adapted readily to current interest rate, in-step with behavior (r = 0.82, P = 1.7 × 10−4; both Medians = 1, n = 61). Lower: Neurons typically reached adaptation criterion within the first sequence (Median = −3, implying adaption within 3 trials before end of first sequence, t60 = −10.17, P = 1.0 × 10−14, one-sample t-test).
© Copyright Policy
Related In: Results  -  Collection

Show All Figures
getmorefigures.php?uid=PMC4340753&req=5

Figure 5: Adaptation dynamics of planning activity, reward proximity control. (a) Sequence-by-sequence adaptation in a single neuron encoding sequence length. Activity changes from spend to save trials (dashed lines) reflected changes in sequence length between successive sequences. Gray curves: sequence-averaged activity (thick line) and trial-by-trial activity (thin line). Green curve: sequence length. Blue curve: within-sequence reward proximity. Arrows: examples for activity changes scaling with sequence length changes. Colored boxes indicate sequences and corresponding lengths. (b) Linear regression of activity of the neuron in (a) on sequence length (left, n = 41), difference in length between subsequent sequences (ΔSequence length, middle, n = 7), and reward proximity (right, n = 41). (c) Population data. Left: sequence value responses (n = 61); activity changes at sequence transitions reflected changes in sequence value (linear regression). Middle: sequence length responses (n = 55); activity changes reflected changes in sequence length. Right: Population activity (sequence value and sequence length responses, n = 116) was unrelated to within-sequence reward proximity. (d) Regression betas for planning activity and reward proximity (n = 116 sequence value and sequence length responses, Kolmogorov-Smirnov test). (e) Behavioral-neuronal adaptation in sequence value neurons. Upper: With a new testing session, planning activity adapted readily to current interest rate, in-step with behavior (r = 0.82, P = 1.7 × 10−4; both Medians = 1, n = 61). Lower: Neurons typically reached adaptation criterion within the first sequence (Median = −3, implying adaption within 3 trials before end of first sequence, t60 = −10.17, P = 1.0 × 10−14, one-sample t-test).
Mentions: If a neuron encoded components of the animals’ saving plan, its activity should update once a sequence is completed and begin to reflect properties of the subsequent sequence. Accordingly, we examined sequence transitions by comparing activity on spend trials and subsequent save trials (the last and first trials of two successive sequences). Figure 5a illustrates such transitions in a single neuron with planning activity related to sequence length. Transitions (dashed vertical lines) were marked by activity changes that scaled with changes in planned sequence length (compare thick gray and green lines). The neuron’s activity reflected planned sequence length within sequences (Fig. 5b left) and changes in planned sequence lengths at transitions (Fig. 5b middle). Activity was unrelated to within-sequence reward proximity (trials until reward, Fig. 5b right). Sequence-by-sequence adaptation was also evident in population activity (Fig. 5c, left and middle). Thus, planning activity adapted sequence-by-sequence to reflect changes in the animals’ internal plan.

Bottom Line: The best rewards are often distant and can only be achieved by planning and decision-making over several steps.Such prospective activity could underlie the formation and pursuit of internal plans characteristic of goal-directed behavior.The existence of neuronal planning activity in the amygdala suggests that this structure is important in guiding behavior toward internally generated, distant goals.

View Article: PubMed Central - PubMed

Affiliation: Department of Physiology, Development and Neuroscience, University of Cambridge, Cambridge, UK.

ABSTRACT
The best rewards are often distant and can only be achieved by planning and decision-making over several steps. We designed a multi-step choice task in which monkeys followed internal plans to save rewards toward self-defined goals. During this self-controlled behavior, amygdala neurons showed future-oriented activity that reflected the animal's plan to obtain specific rewards several trials ahead. This prospective activity encoded crucial components of the animal's plan, including value and length of the planned choice sequence. It began on initial trials when a plan would be formed, reappeared step by step until reward receipt, and readily updated with a new sequence. It predicted performance, including errors, and typically disappeared during instructed behavior. Such prospective activity could underlie the formation and pursuit of internal plans characteristic of goal-directed behavior. The existence of neuronal planning activity in the amygdala suggests that this structure is important in guiding behavior toward internally generated, distant goals.

Show MeSH
Related in: MedlinePlus