Limits...
Modeling the value of strategic actions in the superior colliculus.

Thevarajah D, Webb R, Ferrall C, Dorris MC - Front Behav Neurosci (2010)

Bottom Line: Further, SC activity predicted upcoming choices during the strategic task and upcoming reaction times during the instructed task.Finally, we found that neuronal activity in both tasks correlated with an established learning model, the Experience Weighted Attraction model of action valuation (Camerer and Ho, 1999).Collectively, our results provide evidence that action values hypothesized by learning models are represented in the motor planning regions of the brain in a manner that could be used to select strategic actions.

View Article: PubMed Central - PubMed

Affiliation: Department of Physiology, Centre for Neuroscience Studies and Canadian Institutes of Health Research Group in Sensory-Motor Systems, Queen's University Kingston, ON, Canada.

ABSTRACT
In learning models of strategic game play, an agent constructs a valuation (action value) over possible future choices as a function of past actions and rewards. Choices are then stochastic functions of these action values. Our goal is to uncover a neural signal that correlates with the action value posited by behavioral learning models. We measured activity from neurons in the superior colliculus (SC), a midbrain region involved in planning saccadic eye movements, while monkeys performed two saccade tasks. In the strategic task, monkeys competed against a computer in a saccade version of the mixed-strategy game "matching-pennies". In the instructed task, saccades were elicited through explicit instruction rather than free choices. In both tasks neuronal activity and behavior were shaped by past actions and rewards with more recent events exerting a larger influence. Further, SC activity predicted upcoming choices during the strategic task and upcoming reaction times during the instructed task. Finally, we found that neuronal activity in both tasks correlated with an established learning model, the Experience Weighted Attraction model of action valuation (Camerer and Ho, 1999). Collectively, our results provide evidence that action values hypothesized by learning models are represented in the motor planning regions of the brain in a manner that could be used to select strategic actions.

No MeSH data available.


SCi activity during the current trial.  (A, B) Activity of a representative SCi neuron during the strategic (A) and instructed (B) tasks. Rasters (top panels) and post-synaptic activation functions (bottom panel) are sorted based on saccades directed in (black) and out (gray) of the neuron's response field. The shaded gray bar denotes the epoch during which preparatory activity was analyzed. (C, D) Mean activity of neuron sample in which both strategic (C) and instructed (D) tasks were recorded (38 neurons).
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC2821176&req=5

Figure 4: SCi activity during the current trial. (A, B) Activity of a representative SCi neuron during the strategic (A) and instructed (B) tasks. Rasters (top panels) and post-synaptic activation functions (bottom panel) are sorted based on saccades directed in (black) and out (gray) of the neuron's response field. The shaded gray bar denotes the epoch during which preparatory activity was analyzed. (C, D) Mean activity of neuron sample in which both strategic (C) and instructed (D) tasks were recorded (38 neurons).

Mentions: In both tasks, neuronal activity steadily increased during the warning period in advance of choosing either target (Figure 4). Overall preparatory activity was greater regardless of saccade direction during the strategic task compared to the instructed task (in: p < 0.05, out: p < 0.05). Moreover, in the strategic task activity was segregated for saccades in (99.9 ± 8.8 spikes/s) and out (80.2 ± 7.2 spikes/s, paired t-test, p < 0.001), whereas activity was not segregated between in (63.5 ± 6.5 spikes/s) and out (64.5 ± 6.5 spikes/s) saccades during the instructed task (paired t-test, p > 0.05). This greater overall activation and neuronal selectivity during the strategic task may occur because saccades are under voluntary control and can be planned in advance. In the instructed task the monkey must wait for the presentation of the target.


Modeling the value of strategic actions in the superior colliculus.

Thevarajah D, Webb R, Ferrall C, Dorris MC - Front Behav Neurosci (2010)

SCi activity during the current trial.  (A, B) Activity of a representative SCi neuron during the strategic (A) and instructed (B) tasks. Rasters (top panels) and post-synaptic activation functions (bottom panel) are sorted based on saccades directed in (black) and out (gray) of the neuron's response field. The shaded gray bar denotes the epoch during which preparatory activity was analyzed. (C, D) Mean activity of neuron sample in which both strategic (C) and instructed (D) tasks were recorded (38 neurons).
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC2821176&req=5

Figure 4: SCi activity during the current trial. (A, B) Activity of a representative SCi neuron during the strategic (A) and instructed (B) tasks. Rasters (top panels) and post-synaptic activation functions (bottom panel) are sorted based on saccades directed in (black) and out (gray) of the neuron's response field. The shaded gray bar denotes the epoch during which preparatory activity was analyzed. (C, D) Mean activity of neuron sample in which both strategic (C) and instructed (D) tasks were recorded (38 neurons).
Mentions: In both tasks, neuronal activity steadily increased during the warning period in advance of choosing either target (Figure 4). Overall preparatory activity was greater regardless of saccade direction during the strategic task compared to the instructed task (in: p < 0.05, out: p < 0.05). Moreover, in the strategic task activity was segregated for saccades in (99.9 ± 8.8 spikes/s) and out (80.2 ± 7.2 spikes/s, paired t-test, p < 0.001), whereas activity was not segregated between in (63.5 ± 6.5 spikes/s) and out (64.5 ± 6.5 spikes/s) saccades during the instructed task (paired t-test, p > 0.05). This greater overall activation and neuronal selectivity during the strategic task may occur because saccades are under voluntary control and can be planned in advance. In the instructed task the monkey must wait for the presentation of the target.

Bottom Line: Further, SC activity predicted upcoming choices during the strategic task and upcoming reaction times during the instructed task.Finally, we found that neuronal activity in both tasks correlated with an established learning model, the Experience Weighted Attraction model of action valuation (Camerer and Ho, 1999).Collectively, our results provide evidence that action values hypothesized by learning models are represented in the motor planning regions of the brain in a manner that could be used to select strategic actions.

View Article: PubMed Central - PubMed

Affiliation: Department of Physiology, Centre for Neuroscience Studies and Canadian Institutes of Health Research Group in Sensory-Motor Systems, Queen's University Kingston, ON, Canada.

ABSTRACT
In learning models of strategic game play, an agent constructs a valuation (action value) over possible future choices as a function of past actions and rewards. Choices are then stochastic functions of these action values. Our goal is to uncover a neural signal that correlates with the action value posited by behavioral learning models. We measured activity from neurons in the superior colliculus (SC), a midbrain region involved in planning saccadic eye movements, while monkeys performed two saccade tasks. In the strategic task, monkeys competed against a computer in a saccade version of the mixed-strategy game "matching-pennies". In the instructed task, saccades were elicited through explicit instruction rather than free choices. In both tasks neuronal activity and behavior were shaped by past actions and rewards with more recent events exerting a larger influence. Further, SC activity predicted upcoming choices during the strategic task and upcoming reaction times during the instructed task. Finally, we found that neuronal activity in both tasks correlated with an established learning model, the Experience Weighted Attraction model of action valuation (Camerer and Ho, 1999). Collectively, our results provide evidence that action values hypothesized by learning models are represented in the motor planning regions of the brain in a manner that could be used to select strategic actions.

No MeSH data available.