Limits...
Modeling the value of strategic actions in the superior colliculus.

Thevarajah D, Webb R, Ferrall C, Dorris MC - Front Behav Neurosci (2010)

Bottom Line: Further, SC activity predicted upcoming choices during the strategic task and upcoming reaction times during the instructed task.Finally, we found that neuronal activity in both tasks correlated with an established learning model, the Experience Weighted Attraction model of action valuation (Camerer and Ho, 1999).Collectively, our results provide evidence that action values hypothesized by learning models are represented in the motor planning regions of the brain in a manner that could be used to select strategic actions.

View Article: PubMed Central - PubMed

Affiliation: Department of Physiology, Centre for Neuroscience Studies and Canadian Institutes of Health Research Group in Sensory-Motor Systems, Queen's University Kingston, ON, Canada.

ABSTRACT
In learning models of strategic game play, an agent constructs a valuation (action value) over possible future choices as a function of past actions and rewards. Choices are then stochastic functions of these action values. Our goal is to uncover a neural signal that correlates with the action value posited by behavioral learning models. We measured activity from neurons in the superior colliculus (SC), a midbrain region involved in planning saccadic eye movements, while monkeys performed two saccade tasks. In the strategic task, monkeys competed against a computer in a saccade version of the mixed-strategy game "matching-pennies". In the instructed task, saccades were elicited through explicit instruction rather than free choices. In both tasks neuronal activity and behavior were shaped by past actions and rewards with more recent events exerting a larger influence. Further, SC activity predicted upcoming choices during the strategic task and upcoming reaction times during the instructed task. Finally, we found that neuronal activity in both tasks correlated with an established learning model, the Experience Weighted Attraction model of action valuation (Camerer and Ho, 1999). Collectively, our results provide evidence that action values hypothesized by learning models are represented in the motor planning regions of the brain in a manner that could be used to select strategic actions.

No MeSH data available.


Saccade behavior and SCi activity segregated on previous and future trials.  The data for each trial sequence is presented as the percentage change from the mean data for all trials at trial t (black dots). Note that each data point in the trial sequence represents the influence of an earlier or later trial on the current trial. Therefore, the four colored data points at each time sequence always sum to the mean data point at time t when weighted by the proportion of trials in each category. (A) Changes in in target choices during the strategic task. (B) Changes in in target SRTs during the instructed task. Note that the the ordinate axis has been flipped because SRTs are negatively correlated with SCi activity. (C) Changes in SCi activity during the strategic task. (D) Changes in SCi activity during the instructed task. Filled squares indicate significant differences from the mean activity. Representative standard errors are shown for in/R data points.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC2821176&req=5

Figure 6: Saccade behavior and SCi activity segregated on previous and future trials. The data for each trial sequence is presented as the percentage change from the mean data for all trials at trial t (black dots). Note that each data point in the trial sequence represents the influence of an earlier or later trial on the current trial. Therefore, the four colored data points at each time sequence always sum to the mean data point at time t when weighted by the proportion of trials in each category. (A) Changes in in target choices during the strategic task. (B) Changes in in target SRTs during the instructed task. Note that the the ordinate axis has been flipped because SRTs are negatively correlated with SCi activity. (C) Changes in SCi activity during the strategic task. (D) Changes in SCi activity during the instructed task. Filled squares indicate significant differences from the mean activity. Representative standard errors are shown for in/R data points.

Mentions: To examine any biases exerted by previous saccades and rewards, we segregated SCi activity and saccadic responses on the current trial t based on past (t − n, where 1 ≤ n ≤ 7) and future (t + n, where 1 ≤ n ≤ 3) events (Maljkovic and Nakayama, 1994). Future events were examined for control purposes as these should not exert any influence on the current trial. This sequential analysis is illustrated in Figures 5 and 6 which shows neuronal activity on the current trial segregated into four categories based on four possible events that occurred on the previous trial. (1) a rewarded saccade into the response field (in/R), (2) an unrewarded saccade into the response field (in/U), (3) a rewarded saccade out of the response field (out/R), and (4) an unrewarded saccade out of the response field (out/U).


Modeling the value of strategic actions in the superior colliculus.

Thevarajah D, Webb R, Ferrall C, Dorris MC - Front Behav Neurosci (2010)

Saccade behavior and SCi activity segregated on previous and future trials.  The data for each trial sequence is presented as the percentage change from the mean data for all trials at trial t (black dots). Note that each data point in the trial sequence represents the influence of an earlier or later trial on the current trial. Therefore, the four colored data points at each time sequence always sum to the mean data point at time t when weighted by the proportion of trials in each category. (A) Changes in in target choices during the strategic task. (B) Changes in in target SRTs during the instructed task. Note that the the ordinate axis has been flipped because SRTs are negatively correlated with SCi activity. (C) Changes in SCi activity during the strategic task. (D) Changes in SCi activity during the instructed task. Filled squares indicate significant differences from the mean activity. Representative standard errors are shown for in/R data points.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC2821176&req=5

Figure 6: Saccade behavior and SCi activity segregated on previous and future trials. The data for each trial sequence is presented as the percentage change from the mean data for all trials at trial t (black dots). Note that each data point in the trial sequence represents the influence of an earlier or later trial on the current trial. Therefore, the four colored data points at each time sequence always sum to the mean data point at time t when weighted by the proportion of trials in each category. (A) Changes in in target choices during the strategic task. (B) Changes in in target SRTs during the instructed task. Note that the the ordinate axis has been flipped because SRTs are negatively correlated with SCi activity. (C) Changes in SCi activity during the strategic task. (D) Changes in SCi activity during the instructed task. Filled squares indicate significant differences from the mean activity. Representative standard errors are shown for in/R data points.
Mentions: To examine any biases exerted by previous saccades and rewards, we segregated SCi activity and saccadic responses on the current trial t based on past (t − n, where 1 ≤ n ≤ 7) and future (t + n, where 1 ≤ n ≤ 3) events (Maljkovic and Nakayama, 1994). Future events were examined for control purposes as these should not exert any influence on the current trial. This sequential analysis is illustrated in Figures 5 and 6 which shows neuronal activity on the current trial segregated into four categories based on four possible events that occurred on the previous trial. (1) a rewarded saccade into the response field (in/R), (2) an unrewarded saccade into the response field (in/U), (3) a rewarded saccade out of the response field (out/R), and (4) an unrewarded saccade out of the response field (out/U).

Bottom Line: Further, SC activity predicted upcoming choices during the strategic task and upcoming reaction times during the instructed task.Finally, we found that neuronal activity in both tasks correlated with an established learning model, the Experience Weighted Attraction model of action valuation (Camerer and Ho, 1999).Collectively, our results provide evidence that action values hypothesized by learning models are represented in the motor planning regions of the brain in a manner that could be used to select strategic actions.

View Article: PubMed Central - PubMed

Affiliation: Department of Physiology, Centre for Neuroscience Studies and Canadian Institutes of Health Research Group in Sensory-Motor Systems, Queen's University Kingston, ON, Canada.

ABSTRACT
In learning models of strategic game play, an agent constructs a valuation (action value) over possible future choices as a function of past actions and rewards. Choices are then stochastic functions of these action values. Our goal is to uncover a neural signal that correlates with the action value posited by behavioral learning models. We measured activity from neurons in the superior colliculus (SC), a midbrain region involved in planning saccadic eye movements, while monkeys performed two saccade tasks. In the strategic task, monkeys competed against a computer in a saccade version of the mixed-strategy game "matching-pennies". In the instructed task, saccades were elicited through explicit instruction rather than free choices. In both tasks neuronal activity and behavior were shaped by past actions and rewards with more recent events exerting a larger influence. Further, SC activity predicted upcoming choices during the strategic task and upcoming reaction times during the instructed task. Finally, we found that neuronal activity in both tasks correlated with an established learning model, the Experience Weighted Attraction model of action valuation (Camerer and Ho, 1999). Collectively, our results provide evidence that action values hypothesized by learning models are represented in the motor planning regions of the brain in a manner that could be used to select strategic actions.

No MeSH data available.