Limits...
Planning activity for internally generated reward goals in monkey amygdala neurons.

Hernádi I, Grabenhorst F, Schultz W - Nat. Neurosci. (2015)

Bottom Line: The best rewards are often distant and can only be achieved by planning and decision-making over several steps.Such prospective activity could underlie the formation and pursuit of internal plans characteristic of goal-directed behavior.The existence of neuronal planning activity in the amygdala suggests that this structure is important in guiding behavior toward internally generated, distant goals.

View Article: PubMed Central - PubMed

Affiliation: Department of Physiology, Development and Neuroscience, University of Cambridge, Cambridge, UK.

ABSTRACT
The best rewards are often distant and can only be achieved by planning and decision-making over several steps. We designed a multi-step choice task in which monkeys followed internal plans to save rewards toward self-defined goals. During this self-controlled behavior, amygdala neurons showed future-oriented activity that reflected the animal's plan to obtain specific rewards several trials ahead. This prospective activity encoded crucial components of the animal's plan, including value and length of the planned choice sequence. It began on initial trials when a plan would be formed, reappeared step by step until reward receipt, and readily updated with a new sequence. It predicted performance, including errors, and typically disappeared during instructed behavior. Such prospective activity could underlie the formation and pursuit of internal plans characteristic of goal-directed behavior. The existence of neuronal planning activity in the amygdala suggests that this structure is important in guiding behavior toward internally generated, distant goals.

Show MeSH

Related in: MedlinePlus

Reward-saving behavior in monkeys. (a) Sequential saving task. Animals chose freely to save or spend reward and determined internally the length of each saving sequence. Consecutive save choices increased reward amounts (determined by interest rate); spend choice resulted in reward delivery. Sequences lasted up to 9 consecutive trials (~12 s cycle time/trial). (b) Saving behavior, reward increases, and subjective value functions for different interest rates. Bars: relative frequencies with which animals produced different sequences, combined across animals. Green curves: reward amounts for different sequences. Magenta: subjective values (normalized), combining choice frequencies with reward magnitudes. With highest interest rate, reward stagnated after seven trials; most neuronal recordings involved intermediate interest rates. (c) Monkeys adapted their saving behavior to interest rate. Linear regression of weighted mean sequence length on interest for main task (black, n = 17) and control test with uncued changes in interest (magenta, n = 9). Data combined across animals. (d) Linear regression of reaction time on final sequence length. Reaction times (equally populated bins pooled over animals and interest rates, z-normalized within sessions) on spend trials (black, averaged over n = 3,033 trials) and save trials (magenta, averaged over n = 8,500 trials) were shorter for longer sequences (i.e. higher rewards). (e) Logistic regression of trial-by-trial choices. Spend/save value: subjective value associated with spending/saving on current trial; sequence value: subjective sequence value (spend value on final trial). Bias: constant; Cue position: left/right save cue position; Juice/day: consumed juice; Monkey: animal identity. **P < 0.005, *P < 0.05; n.s. not significant. Error bars: s.e.m.
© Copyright Policy
Related In: Results  -  Collection


getmorefigures.php?uid=PMC4340753&req=5

Figure 1: Reward-saving behavior in monkeys. (a) Sequential saving task. Animals chose freely to save or spend reward and determined internally the length of each saving sequence. Consecutive save choices increased reward amounts (determined by interest rate); spend choice resulted in reward delivery. Sequences lasted up to 9 consecutive trials (~12 s cycle time/trial). (b) Saving behavior, reward increases, and subjective value functions for different interest rates. Bars: relative frequencies with which animals produced different sequences, combined across animals. Green curves: reward amounts for different sequences. Magenta: subjective values (normalized), combining choice frequencies with reward magnitudes. With highest interest rate, reward stagnated after seven trials; most neuronal recordings involved intermediate interest rates. (c) Monkeys adapted their saving behavior to interest rate. Linear regression of weighted mean sequence length on interest for main task (black, n = 17) and control test with uncued changes in interest (magenta, n = 9). Data combined across animals. (d) Linear regression of reaction time on final sequence length. Reaction times (equally populated bins pooled over animals and interest rates, z-normalized within sessions) on spend trials (black, averaged over n = 3,033 trials) and save trials (magenta, averaged over n = 8,500 trials) were shorter for longer sequences (i.e. higher rewards). (e) Logistic regression of trial-by-trial choices. Spend/save value: subjective value associated with spending/saving on current trial; sequence value: subjective sequence value (spend value on final trial). Bias: constant; Cue position: left/right save cue position; Juice/day: consumed juice; Monkey: animal identity. **P < 0.005, *P < 0.05; n.s. not significant. Error bars: s.e.m.

Mentions: Two monkeys performed in a sequential reward-saving task in which they could follow internal plans towards obtaining reward at the end of a sequence of trials. On each trial (i.e. each step within a sequence) the animals freely chose to save juice reward for future consumption or spend the already saved amount (Fig. 1a). Consecutive save choices increased the available juice amount as determined by a given ‘interest rate’ (Fig. 1b, green, Eq. 1). Choices were made by a saccade towards the save or spend cue; pre-trained save cues indicated current interest rate. The animals freely determined the length of each saving sequence. This self-controlled and sequential task design allowed the animals to plan their behavior over multiple trials and anticipate final rewards more than 100 s in advance (up to 9 consecutive trials with ~12 s cycle time). Randomized cue positions precluded planning of left-right action sequences. To confirm the internal nature of planning, we also tested externally instructed ‘imperative’ save-spend sequences with comparable lengths.


Planning activity for internally generated reward goals in monkey amygdala neurons.

Hernádi I, Grabenhorst F, Schultz W - Nat. Neurosci. (2015)

Reward-saving behavior in monkeys. (a) Sequential saving task. Animals chose freely to save or spend reward and determined internally the length of each saving sequence. Consecutive save choices increased reward amounts (determined by interest rate); spend choice resulted in reward delivery. Sequences lasted up to 9 consecutive trials (~12 s cycle time/trial). (b) Saving behavior, reward increases, and subjective value functions for different interest rates. Bars: relative frequencies with which animals produced different sequences, combined across animals. Green curves: reward amounts for different sequences. Magenta: subjective values (normalized), combining choice frequencies with reward magnitudes. With highest interest rate, reward stagnated after seven trials; most neuronal recordings involved intermediate interest rates. (c) Monkeys adapted their saving behavior to interest rate. Linear regression of weighted mean sequence length on interest for main task (black, n = 17) and control test with uncued changes in interest (magenta, n = 9). Data combined across animals. (d) Linear regression of reaction time on final sequence length. Reaction times (equally populated bins pooled over animals and interest rates, z-normalized within sessions) on spend trials (black, averaged over n = 3,033 trials) and save trials (magenta, averaged over n = 8,500 trials) were shorter for longer sequences (i.e. higher rewards). (e) Logistic regression of trial-by-trial choices. Spend/save value: subjective value associated with spending/saving on current trial; sequence value: subjective sequence value (spend value on final trial). Bias: constant; Cue position: left/right save cue position; Juice/day: consumed juice; Monkey: animal identity. **P < 0.005, *P < 0.05; n.s. not significant. Error bars: s.e.m.
© Copyright Policy
Related In: Results  -  Collection

Show All Figures
getmorefigures.php?uid=PMC4340753&req=5

Figure 1: Reward-saving behavior in monkeys. (a) Sequential saving task. Animals chose freely to save or spend reward and determined internally the length of each saving sequence. Consecutive save choices increased reward amounts (determined by interest rate); spend choice resulted in reward delivery. Sequences lasted up to 9 consecutive trials (~12 s cycle time/trial). (b) Saving behavior, reward increases, and subjective value functions for different interest rates. Bars: relative frequencies with which animals produced different sequences, combined across animals. Green curves: reward amounts for different sequences. Magenta: subjective values (normalized), combining choice frequencies with reward magnitudes. With highest interest rate, reward stagnated after seven trials; most neuronal recordings involved intermediate interest rates. (c) Monkeys adapted their saving behavior to interest rate. Linear regression of weighted mean sequence length on interest for main task (black, n = 17) and control test with uncued changes in interest (magenta, n = 9). Data combined across animals. (d) Linear regression of reaction time on final sequence length. Reaction times (equally populated bins pooled over animals and interest rates, z-normalized within sessions) on spend trials (black, averaged over n = 3,033 trials) and save trials (magenta, averaged over n = 8,500 trials) were shorter for longer sequences (i.e. higher rewards). (e) Logistic regression of trial-by-trial choices. Spend/save value: subjective value associated with spending/saving on current trial; sequence value: subjective sequence value (spend value on final trial). Bias: constant; Cue position: left/right save cue position; Juice/day: consumed juice; Monkey: animal identity. **P < 0.005, *P < 0.05; n.s. not significant. Error bars: s.e.m.
Mentions: Two monkeys performed in a sequential reward-saving task in which they could follow internal plans towards obtaining reward at the end of a sequence of trials. On each trial (i.e. each step within a sequence) the animals freely chose to save juice reward for future consumption or spend the already saved amount (Fig. 1a). Consecutive save choices increased the available juice amount as determined by a given ‘interest rate’ (Fig. 1b, green, Eq. 1). Choices were made by a saccade towards the save or spend cue; pre-trained save cues indicated current interest rate. The animals freely determined the length of each saving sequence. This self-controlled and sequential task design allowed the animals to plan their behavior over multiple trials and anticipate final rewards more than 100 s in advance (up to 9 consecutive trials with ~12 s cycle time). Randomized cue positions precluded planning of left-right action sequences. To confirm the internal nature of planning, we also tested externally instructed ‘imperative’ save-spend sequences with comparable lengths.

Bottom Line: The best rewards are often distant and can only be achieved by planning and decision-making over several steps.Such prospective activity could underlie the formation and pursuit of internal plans characteristic of goal-directed behavior.The existence of neuronal planning activity in the amygdala suggests that this structure is important in guiding behavior toward internally generated, distant goals.

View Article: PubMed Central - PubMed

Affiliation: Department of Physiology, Development and Neuroscience, University of Cambridge, Cambridge, UK.

ABSTRACT
The best rewards are often distant and can only be achieved by planning and decision-making over several steps. We designed a multi-step choice task in which monkeys followed internal plans to save rewards toward self-defined goals. During this self-controlled behavior, amygdala neurons showed future-oriented activity that reflected the animal's plan to obtain specific rewards several trials ahead. This prospective activity encoded crucial components of the animal's plan, including value and length of the planned choice sequence. It began on initial trials when a plan would be formed, reappeared step by step until reward receipt, and readily updated with a new sequence. It predicted performance, including errors, and typically disappeared during instructed behavior. Such prospective activity could underlie the formation and pursuit of internal plans characteristic of goal-directed behavior. The existence of neuronal planning activity in the amygdala suggests that this structure is important in guiding behavior toward internally generated, distant goals.

Show MeSH
Related in: MedlinePlus