Limits...
Reward Pays the Cost of Noise Reduction in Motor and Cognitive Control.

Manohar SG, Chong TT, Apps MA, Batla A, Stamelou M, Jarman PR, Bhatia KP, Husain M - Curr. Biol. (2015)

Bottom Line: Both faster speeds and smaller errors were observed with higher incentives, with the results best fitted by a model including a precision cost.Recent theories consider dopamine to be a key neuromodulator in mediating motivational effects of reward.On this view, the pattern of reduced reward sensitivity in PD patients can specifically be accounted for by a higher cost for controlling noise.

View Article: PubMed Central - PubMed

Affiliation: Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, Oxford OX3 9DU, UK; Department of Experimental Psychology, University of Oxford, Oxford OX1 3UD, UK; Institute of Neurology, University College London, London WC1N 3BG, UK; Institute of Cognitive Neuroscience, University College London, London WC1N 3AR, UK; National Hospital for Neurology and Neurosurgery, Queen Square, London WC1N 3BG, UK. Electronic address: sanjay.manohar@ndcn.ox.ac.uk.

No MeSH data available.


Related in: MedlinePlus

Optimal Control Model to Explain the Effect of Reward IncentivesIn order to account for the ability of reward to improve both speed and accuracy, we hypothesized that in addition to a “vigor” or force signal (uF) that determines a movement’s speed, individuals are also able to select a “precision” signal (uP) that determines the amount of variability in a movement. Crucially, this precision signal is also costly.(A) Each given motor command, i.e., a pair of force and precision u = (uF, uP), has an EV. The image shows EV as a function of u, with the best combination as blue and worst as red. The value depends on three effects. First, the reward available is temporally discounted by the time taken by the movement, e.g., by hyperbolic discounting . Second, this reward is only obtained if the movement is on target. We assume a Gaussian variation Φ of the endpoint proportional to the size of the motor command. Third, although we can go faster to reduce temporal discounting (increasing uF) and be more precise to reduce error (increasing uP), both of these incur a cost proportional to the squared control signal, u2. This leads to an optimal combination of force and precision for each movement, u∗.(B) The optimal motor command for a situation depends on the reward level R and on two subject-specific parameters: the discount rate k and the noise-control cost σ. The optimal precision (upper panels) and force (lower panels) both increase with increasing reward (y axis), indicating that reward induces greater “spending” on both speed and accuracy. However, precision and force are differentially influenced by reward, and the balance depends on the urgency (temporal discount, k, left panels) and error constraints (encapsulated by σ, right panels).(C) The optimal commands determine the velocity and duration of each movement and the amount of variability for a desired movement amplitude. Reward always increases velocity (lower panels). However, variability may increase or decrease with reward (upper panels), depending on σ and k. A subject with minimal discounting (e.g., k < 0.5) becomes less variable with higher reward, whereas a subject with high discount rates (e.g., k > 1) in fact tends to become more variable with higher reward (upper panels) as they are under greater time pressure, i.e., trading speed for accuracy. These effects are re-plotted on different axes in Figure S5.
© Copyright Policy - CC BY
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4557747&req=5

fig3: Optimal Control Model to Explain the Effect of Reward IncentivesIn order to account for the ability of reward to improve both speed and accuracy, we hypothesized that in addition to a “vigor” or force signal (uF) that determines a movement’s speed, individuals are also able to select a “precision” signal (uP) that determines the amount of variability in a movement. Crucially, this precision signal is also costly.(A) Each given motor command, i.e., a pair of force and precision u = (uF, uP), has an EV. The image shows EV as a function of u, with the best combination as blue and worst as red. The value depends on three effects. First, the reward available is temporally discounted by the time taken by the movement, e.g., by hyperbolic discounting . Second, this reward is only obtained if the movement is on target. We assume a Gaussian variation Φ of the endpoint proportional to the size of the motor command. Third, although we can go faster to reduce temporal discounting (increasing uF) and be more precise to reduce error (increasing uP), both of these incur a cost proportional to the squared control signal, u2. This leads to an optimal combination of force and precision for each movement, u∗.(B) The optimal motor command for a situation depends on the reward level R and on two subject-specific parameters: the discount rate k and the noise-control cost σ. The optimal precision (upper panels) and force (lower panels) both increase with increasing reward (y axis), indicating that reward induces greater “spending” on both speed and accuracy. However, precision and force are differentially influenced by reward, and the balance depends on the urgency (temporal discount, k, left panels) and error constraints (encapsulated by σ, right panels).(C) The optimal commands determine the velocity and duration of each movement and the amount of variability for a desired movement amplitude. Reward always increases velocity (lower panels). However, variability may increase or decrease with reward (upper panels), depending on σ and k. A subject with minimal discounting (e.g., k < 0.5) becomes less variable with higher reward, whereas a subject with high discount rates (e.g., k > 1) in fact tends to become more variable with higher reward (upper panels) as they are under greater time pressure, i.e., trading speed for accuracy. These effects are re-plotted on different axes in Figure S5.

Mentions: When motor noise and accuracy are made irrelevant (e.g., for very large targets), then increasing reward simply increases the cost of time relative to energetic costs. Subjects are consequently more willing to exert more effort to move faster, so higher reward increases optimal speed (Figure 2A), as in the orthodox view [16, 36]. Conversely, if speed is ignored and only accuracy and precision are considered, then a new trade-off occurs between the cost of precision and the cost of errors. Since precision improves the probability of success but is expensive, there is an optimal level of accuracy which increases when more reward is on offer (Figure 2B). Crucially, when both precision and force are allowed to vary simultaneously, reward has the effect of increasing the optimal velocity while also reducing motor variability (Figure 2C). For each reward level R, there is a particular combination of force uF and precision uP that maximizes EV (Figures 3A and 3B), corresponding to an optimal saccade velocity and endpoint variability (Figures 3C and S5). The optimum will depend on an individual’s temporal discount rate k and the noise parameter σ. To account for the possibility that not all noise may be controllable by a system (e.g., noise in the effector itself), an additive baseline noise term σ0 can be included. In this case, σ0 represents a participant’s fixed motor noise, whereas σ represents the relative cost of precision, compared to energetic (force) cost.


Reward Pays the Cost of Noise Reduction in Motor and Cognitive Control.

Manohar SG, Chong TT, Apps MA, Batla A, Stamelou M, Jarman PR, Bhatia KP, Husain M - Curr. Biol. (2015)

Optimal Control Model to Explain the Effect of Reward IncentivesIn order to account for the ability of reward to improve both speed and accuracy, we hypothesized that in addition to a “vigor” or force signal (uF) that determines a movement’s speed, individuals are also able to select a “precision” signal (uP) that determines the amount of variability in a movement. Crucially, this precision signal is also costly.(A) Each given motor command, i.e., a pair of force and precision u = (uF, uP), has an EV. The image shows EV as a function of u, with the best combination as blue and worst as red. The value depends on three effects. First, the reward available is temporally discounted by the time taken by the movement, e.g., by hyperbolic discounting . Second, this reward is only obtained if the movement is on target. We assume a Gaussian variation Φ of the endpoint proportional to the size of the motor command. Third, although we can go faster to reduce temporal discounting (increasing uF) and be more precise to reduce error (increasing uP), both of these incur a cost proportional to the squared control signal, u2. This leads to an optimal combination of force and precision for each movement, u∗.(B) The optimal motor command for a situation depends on the reward level R and on two subject-specific parameters: the discount rate k and the noise-control cost σ. The optimal precision (upper panels) and force (lower panels) both increase with increasing reward (y axis), indicating that reward induces greater “spending” on both speed and accuracy. However, precision and force are differentially influenced by reward, and the balance depends on the urgency (temporal discount, k, left panels) and error constraints (encapsulated by σ, right panels).(C) The optimal commands determine the velocity and duration of each movement and the amount of variability for a desired movement amplitude. Reward always increases velocity (lower panels). However, variability may increase or decrease with reward (upper panels), depending on σ and k. A subject with minimal discounting (e.g., k < 0.5) becomes less variable with higher reward, whereas a subject with high discount rates (e.g., k > 1) in fact tends to become more variable with higher reward (upper panels) as they are under greater time pressure, i.e., trading speed for accuracy. These effects are re-plotted on different axes in Figure S5.
© Copyright Policy - CC BY
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4557747&req=5

fig3: Optimal Control Model to Explain the Effect of Reward IncentivesIn order to account for the ability of reward to improve both speed and accuracy, we hypothesized that in addition to a “vigor” or force signal (uF) that determines a movement’s speed, individuals are also able to select a “precision” signal (uP) that determines the amount of variability in a movement. Crucially, this precision signal is also costly.(A) Each given motor command, i.e., a pair of force and precision u = (uF, uP), has an EV. The image shows EV as a function of u, with the best combination as blue and worst as red. The value depends on three effects. First, the reward available is temporally discounted by the time taken by the movement, e.g., by hyperbolic discounting . Second, this reward is only obtained if the movement is on target. We assume a Gaussian variation Φ of the endpoint proportional to the size of the motor command. Third, although we can go faster to reduce temporal discounting (increasing uF) and be more precise to reduce error (increasing uP), both of these incur a cost proportional to the squared control signal, u2. This leads to an optimal combination of force and precision for each movement, u∗.(B) The optimal motor command for a situation depends on the reward level R and on two subject-specific parameters: the discount rate k and the noise-control cost σ. The optimal precision (upper panels) and force (lower panels) both increase with increasing reward (y axis), indicating that reward induces greater “spending” on both speed and accuracy. However, precision and force are differentially influenced by reward, and the balance depends on the urgency (temporal discount, k, left panels) and error constraints (encapsulated by σ, right panels).(C) The optimal commands determine the velocity and duration of each movement and the amount of variability for a desired movement amplitude. Reward always increases velocity (lower panels). However, variability may increase or decrease with reward (upper panels), depending on σ and k. A subject with minimal discounting (e.g., k < 0.5) becomes less variable with higher reward, whereas a subject with high discount rates (e.g., k > 1) in fact tends to become more variable with higher reward (upper panels) as they are under greater time pressure, i.e., trading speed for accuracy. These effects are re-plotted on different axes in Figure S5.
Mentions: When motor noise and accuracy are made irrelevant (e.g., for very large targets), then increasing reward simply increases the cost of time relative to energetic costs. Subjects are consequently more willing to exert more effort to move faster, so higher reward increases optimal speed (Figure 2A), as in the orthodox view [16, 36]. Conversely, if speed is ignored and only accuracy and precision are considered, then a new trade-off occurs between the cost of precision and the cost of errors. Since precision improves the probability of success but is expensive, there is an optimal level of accuracy which increases when more reward is on offer (Figure 2B). Crucially, when both precision and force are allowed to vary simultaneously, reward has the effect of increasing the optimal velocity while also reducing motor variability (Figure 2C). For each reward level R, there is a particular combination of force uF and precision uP that maximizes EV (Figures 3A and 3B), corresponding to an optimal saccade velocity and endpoint variability (Figures 3C and S5). The optimum will depend on an individual’s temporal discount rate k and the noise parameter σ. To account for the possibility that not all noise may be controllable by a system (e.g., noise in the effector itself), an additive baseline noise term σ0 can be included. In this case, σ0 represents a participant’s fixed motor noise, whereas σ represents the relative cost of precision, compared to energetic (force) cost.

Bottom Line: Both faster speeds and smaller errors were observed with higher incentives, with the results best fitted by a model including a precision cost.Recent theories consider dopamine to be a key neuromodulator in mediating motivational effects of reward.On this view, the pattern of reduced reward sensitivity in PD patients can specifically be accounted for by a higher cost for controlling noise.

View Article: PubMed Central - PubMed

Affiliation: Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, Oxford OX3 9DU, UK; Department of Experimental Psychology, University of Oxford, Oxford OX1 3UD, UK; Institute of Neurology, University College London, London WC1N 3BG, UK; Institute of Cognitive Neuroscience, University College London, London WC1N 3AR, UK; National Hospital for Neurology and Neurosurgery, Queen Square, London WC1N 3BG, UK. Electronic address: sanjay.manohar@ndcn.ox.ac.uk.

No MeSH data available.


Related in: MedlinePlus