Limits...
Undesirable Choice Biases with Small Differences in the Spatial Structure of Chance Stimulus Sequences.

Herrera D, Treviño M - PLoS ONE (2015)

Bottom Line: We compared the choice patterns produced by a 'Rational Decision Maker' (RDM) in response to computer-generated FRS and GLS training sequences.Thus, discrete changes in the training paradigms did not translate linearly into modifications in the pattern of choices generated by a RDM.Virtual RDMs could be further employed to guide the selection of proper training schedules for perceptual decision-making studies.

View Article: PubMed Central - PubMed

Affiliation: Instituto de Neurociencias, Universidad de Guadalajara, Guadalajara, Jalisco, México.

ABSTRACT
In two-alternative discrimination tasks, experimenters usually randomize the location of the rewarded stimulus so that systematic behavior with respect to irrelevant stimuli can only produce chance performance on the learning curves. One way to achieve this is to use random numbers derived from a discrete binomial distribution to create a 'full random training schedule' (FRS). When using FRS, however, sporadic but long laterally-biased training sequences occur by chance and such 'input biases' are thought to promote the generation of laterally-biased choices (i.e., 'output biases'). As an alternative, a 'Gellerman-like training schedule' (GLS) can be used. It removes most input biases by prohibiting the reward from appearing on the same location for more than three consecutive trials. The sequence of past rewards obtained from choosing a particular discriminative stimulus influences the probability of choosing that same stimulus on subsequent trials. Assuming that the long-term average ratio of choices matches the long-term average ratio of reinforcers, we hypothesized that a reduced amount of input biases in GLS compared to FRS should lead to a reduced production of output biases. We compared the choice patterns produced by a 'Rational Decision Maker' (RDM) in response to computer-generated FRS and GLS training sequences. To create a virtual RDM, we implemented an algorithm that generated choices based on past rewards. Our simulations revealed that, although the GLS presented fewer input biases than the FRS, the virtual RDM produced more output biases with GLS than with FRS under a variety of test conditions. Our results reveal that the statistical and temporal properties of training sequences interacted with the RDM to influence the production of output biases. Thus, discrete changes in the training paradigms did not translate linearly into modifications in the pattern of choices generated by a RDM. Virtual RDMs could be further employed to guide the selection of proper training schedules for perceptual decision-making studies.

No MeSH data available.


Related in: MedlinePlus

Virtual decision maker and training schedules with different input bias probabilities.(A) A training sequence with a % input biases is processed by a virtual 'Rational Decision Maker' (RDM) to produce choice sequences with a % output biases. (B) The RDM algorithm varied according to a sigmoidal function, in which the probability of choosing to the right (PR, y-axis) depended on the weight function for the right option (VR, x-axis) and on the slope of the curve (β; darker lines represent higher β values; Eq 2). (C) The probability of finding laterally biased sequences (y-axis) of different lengths (x-axis) is smaller using GLS (gray line) than FRS (black line). The opposite occurred for alternating sequences (inset). The differences between group probabilities are displayed by dotted lines and are always negative. We calculated the probabilities for biased and alternating sequences by creating independent sets of randomized sequences. Average probabilities for biased/alternating 2-trial-sequences with the FRS schedule converge towards 50% when the number of trials is increased (v.gr. from 100 to 1000; not illustrated).
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4549334&req=5

pone.0136084.g001: Virtual decision maker and training schedules with different input bias probabilities.(A) A training sequence with a % input biases is processed by a virtual 'Rational Decision Maker' (RDM) to produce choice sequences with a % output biases. (B) The RDM algorithm varied according to a sigmoidal function, in which the probability of choosing to the right (PR, y-axis) depended on the weight function for the right option (VR, x-axis) and on the slope of the curve (β; darker lines represent higher β values; Eq 2). (C) The probability of finding laterally biased sequences (y-axis) of different lengths (x-axis) is smaller using GLS (gray line) than FRS (black line). The opposite occurred for alternating sequences (inset). The differences between group probabilities are displayed by dotted lines and are always negative. We calculated the probabilities for biased and alternating sequences by creating independent sets of randomized sequences. Average probabilities for biased/alternating 2-trial-sequences with the FRS schedule converge towards 50% when the number of trials is increased (v.gr. from 100 to 1000; not illustrated).

Mentions: Our specific aim was to compare the choice patterns produced by a RDM in response to FRS and GLS training schedules (Fig 1A). First, we made linear arrays of pseudo-random numbers derived from a discrete binomial distribution to create FRS and GLS training schedules (100 trials x 100 repetitions). The GLS algorithm was identical to the FRS one, but it additionally prohibited sequence repetitions of more than three consecutive trials [2–4,14]. Reinforcement was provided on each trial for one of the two mutually exclusive options (i.e. probability of reward per trial = 100%; probability of reward per side = 50%). To create a virtual RDM, we implemented an algorithm that made choices based on past rewards and choices according to an exponentially-weighted moving average filter (EWMA), but was insensitive to differences in discriminative input signals [1]. With this model, we assumed that the integration of past rewards is imperfect (i.e. leaky), which translates into a finite effective memory on estimates of income, making them local rather than global (in time) [13]. Indeed, the EWMA provides a description for short-term memory in which a reinforcer produces smaller effects into current choices as one considers responses that extend further into the past [12,22]. We chose to use the EWMA based on a series of quantitative observations made on the choice records from diverse animal models [4,11,13,19,23], and because it has fewer free parameters than other alternative models v.gr. [24]. 'Memory gradients' have been also described with hyperbolic functions [19,25], but this won't be addressed here.


Undesirable Choice Biases with Small Differences in the Spatial Structure of Chance Stimulus Sequences.

Herrera D, Treviño M - PLoS ONE (2015)

Virtual decision maker and training schedules with different input bias probabilities.(A) A training sequence with a % input biases is processed by a virtual 'Rational Decision Maker' (RDM) to produce choice sequences with a % output biases. (B) The RDM algorithm varied according to a sigmoidal function, in which the probability of choosing to the right (PR, y-axis) depended on the weight function for the right option (VR, x-axis) and on the slope of the curve (β; darker lines represent higher β values; Eq 2). (C) The probability of finding laterally biased sequences (y-axis) of different lengths (x-axis) is smaller using GLS (gray line) than FRS (black line). The opposite occurred for alternating sequences (inset). The differences between group probabilities are displayed by dotted lines and are always negative. We calculated the probabilities for biased and alternating sequences by creating independent sets of randomized sequences. Average probabilities for biased/alternating 2-trial-sequences with the FRS schedule converge towards 50% when the number of trials is increased (v.gr. from 100 to 1000; not illustrated).
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4549334&req=5

pone.0136084.g001: Virtual decision maker and training schedules with different input bias probabilities.(A) A training sequence with a % input biases is processed by a virtual 'Rational Decision Maker' (RDM) to produce choice sequences with a % output biases. (B) The RDM algorithm varied according to a sigmoidal function, in which the probability of choosing to the right (PR, y-axis) depended on the weight function for the right option (VR, x-axis) and on the slope of the curve (β; darker lines represent higher β values; Eq 2). (C) The probability of finding laterally biased sequences (y-axis) of different lengths (x-axis) is smaller using GLS (gray line) than FRS (black line). The opposite occurred for alternating sequences (inset). The differences between group probabilities are displayed by dotted lines and are always negative. We calculated the probabilities for biased and alternating sequences by creating independent sets of randomized sequences. Average probabilities for biased/alternating 2-trial-sequences with the FRS schedule converge towards 50% when the number of trials is increased (v.gr. from 100 to 1000; not illustrated).
Mentions: Our specific aim was to compare the choice patterns produced by a RDM in response to FRS and GLS training schedules (Fig 1A). First, we made linear arrays of pseudo-random numbers derived from a discrete binomial distribution to create FRS and GLS training schedules (100 trials x 100 repetitions). The GLS algorithm was identical to the FRS one, but it additionally prohibited sequence repetitions of more than three consecutive trials [2–4,14]. Reinforcement was provided on each trial for one of the two mutually exclusive options (i.e. probability of reward per trial = 100%; probability of reward per side = 50%). To create a virtual RDM, we implemented an algorithm that made choices based on past rewards and choices according to an exponentially-weighted moving average filter (EWMA), but was insensitive to differences in discriminative input signals [1]. With this model, we assumed that the integration of past rewards is imperfect (i.e. leaky), which translates into a finite effective memory on estimates of income, making them local rather than global (in time) [13]. Indeed, the EWMA provides a description for short-term memory in which a reinforcer produces smaller effects into current choices as one considers responses that extend further into the past [12,22]. We chose to use the EWMA based on a series of quantitative observations made on the choice records from diverse animal models [4,11,13,19,23], and because it has fewer free parameters than other alternative models v.gr. [24]. 'Memory gradients' have been also described with hyperbolic functions [19,25], but this won't be addressed here.

Bottom Line: We compared the choice patterns produced by a 'Rational Decision Maker' (RDM) in response to computer-generated FRS and GLS training sequences.Thus, discrete changes in the training paradigms did not translate linearly into modifications in the pattern of choices generated by a RDM.Virtual RDMs could be further employed to guide the selection of proper training schedules for perceptual decision-making studies.

View Article: PubMed Central - PubMed

Affiliation: Instituto de Neurociencias, Universidad de Guadalajara, Guadalajara, Jalisco, México.

ABSTRACT
In two-alternative discrimination tasks, experimenters usually randomize the location of the rewarded stimulus so that systematic behavior with respect to irrelevant stimuli can only produce chance performance on the learning curves. One way to achieve this is to use random numbers derived from a discrete binomial distribution to create a 'full random training schedule' (FRS). When using FRS, however, sporadic but long laterally-biased training sequences occur by chance and such 'input biases' are thought to promote the generation of laterally-biased choices (i.e., 'output biases'). As an alternative, a 'Gellerman-like training schedule' (GLS) can be used. It removes most input biases by prohibiting the reward from appearing on the same location for more than three consecutive trials. The sequence of past rewards obtained from choosing a particular discriminative stimulus influences the probability of choosing that same stimulus on subsequent trials. Assuming that the long-term average ratio of choices matches the long-term average ratio of reinforcers, we hypothesized that a reduced amount of input biases in GLS compared to FRS should lead to a reduced production of output biases. We compared the choice patterns produced by a 'Rational Decision Maker' (RDM) in response to computer-generated FRS and GLS training sequences. To create a virtual RDM, we implemented an algorithm that generated choices based on past rewards. Our simulations revealed that, although the GLS presented fewer input biases than the FRS, the virtual RDM produced more output biases with GLS than with FRS under a variety of test conditions. Our results reveal that the statistical and temporal properties of training sequences interacted with the RDM to influence the production of output biases. Thus, discrete changes in the training paradigms did not translate linearly into modifications in the pattern of choices generated by a RDM. Virtual RDMs could be further employed to guide the selection of proper training schedules for perceptual decision-making studies.

No MeSH data available.


Related in: MedlinePlus