Limits...
Model-based learning protects against forming habits.

Gillan CM, Otto AR, Phelps EA, Daw ND - Cogn Affect Behav Neurosci (2015)

Bottom Line: Studies in humans and rodents have suggested that behavior can at times be "goal-directed"-that is, planned, and purposeful-and at times "habitual"-that is, inflexible and automatically evoked by stimuli.We then tested for habits by devaluing one of the rewards that had reinforced behavior.In each case, we found that individual differences in model-based learning predicted the participants' subsequent sensitivity to outcome devaluation, suggesting that an associative mechanism underlies a bias toward habit formation in healthy individuals.

View Article: PubMed Central - PubMed

Affiliation: Department of Psychology, New York University, 6 Washington Place, New York, NY, 10003, USA, claire.gillan@gmail.com.

ABSTRACT
Studies in humans and rodents have suggested that behavior can at times be "goal-directed"-that is, planned, and purposeful-and at times "habitual"-that is, inflexible and automatically evoked by stimuli. This distinction is central to conceptions of pathological compulsion, as in drug abuse and obsessive-compulsive disorder. Evidence for the distinction has primarily come from outcome devaluation studies, in which the sensitivity of a previously learned behavior to motivational change is used to assay the dominance of habits versus goal-directed actions. However, little is known about how habits and goal-directed control arise. Specifically, in the present study we sought to reveal the trial-by-trial dynamics of instrumental learning that would promote, and protect against, developing habits. In two complementary experiments with independent samples, participants completed a sequential decision task that dissociated two computational-learning mechanisms, model-based and model-free. We then tested for habits by devaluing one of the rewards that had reinforced behavior. In each case, we found that individual differences in model-based learning predicted the participants' subsequent sensitivity to outcome devaluation, suggesting that an associative mechanism underlies a bias toward habit formation in healthy individuals.

No MeSH data available.


Related in: MedlinePlus

Experiment 1: Effect sizes (beta weights) from the logistic regression model (Table 1). Significant effects were observed for reward (model-free, p < .001), the Reward × Transition interaction (model-based, p = .020), and the predicted three-way interaction of reward, transition, and devaluation sensitivity (p = .003). rew = reward, trans = transition, dev = devaluation sensitivity
© Copyright Policy - OpenAccess
Related In: Results  -  Collection


getmorefigures.php?uid=PMC4526597&req=5

Fig3: Experiment 1: Effect sizes (beta weights) from the logistic regression model (Table 1). Significant effects were observed for reward (model-free, p < .001), the Reward × Transition interaction (model-based, p = .020), and the predicted three-way interaction of reward, transition, and devaluation sensitivity (p = .003). rew = reward, trans = transition, dev = devaluation sensitivity

Mentions: To assess whether model-based or model-free learning strategies were associated with the formation of devaluation-insensitive habits, we tested for the presence of (1) Reward × Devaluation and (2) Reward × Transition × Devaluation interactions. These significant interactions would indicate that a relationship existed between habits and the strengths of model-free and model-based learning, respectively. We found evidence in support of the latter hypothesis, such that participants who were more model-based during training also showed larger goal-directed sensitivity to devaluation in the habit test, (β = 0.1, standard error [SE] = 0.03, p = .003; see Table 1 and Fig. 3). No such relationship was seen for model-free responding. Since devaluation sensitivity scores were standardized for inclusion in the regression model, the estimated coefficients in Table 1 imply that an increase of one standard deviation in devaluation sensitivity doubles the observable effect of model-based learning, whereas if devaluation sensitivity is one standard deviation below the mean, it eliminates model-based learning altogether.Fig. 3


Model-based learning protects against forming habits.

Gillan CM, Otto AR, Phelps EA, Daw ND - Cogn Affect Behav Neurosci (2015)

Experiment 1: Effect sizes (beta weights) from the logistic regression model (Table 1). Significant effects were observed for reward (model-free, p < .001), the Reward × Transition interaction (model-based, p = .020), and the predicted three-way interaction of reward, transition, and devaluation sensitivity (p = .003). rew = reward, trans = transition, dev = devaluation sensitivity
© Copyright Policy - OpenAccess
Related In: Results  -  Collection

Show All Figures
getmorefigures.php?uid=PMC4526597&req=5

Fig3: Experiment 1: Effect sizes (beta weights) from the logistic regression model (Table 1). Significant effects were observed for reward (model-free, p < .001), the Reward × Transition interaction (model-based, p = .020), and the predicted three-way interaction of reward, transition, and devaluation sensitivity (p = .003). rew = reward, trans = transition, dev = devaluation sensitivity
Mentions: To assess whether model-based or model-free learning strategies were associated with the formation of devaluation-insensitive habits, we tested for the presence of (1) Reward × Devaluation and (2) Reward × Transition × Devaluation interactions. These significant interactions would indicate that a relationship existed between habits and the strengths of model-free and model-based learning, respectively. We found evidence in support of the latter hypothesis, such that participants who were more model-based during training also showed larger goal-directed sensitivity to devaluation in the habit test, (β = 0.1, standard error [SE] = 0.03, p = .003; see Table 1 and Fig. 3). No such relationship was seen for model-free responding. Since devaluation sensitivity scores were standardized for inclusion in the regression model, the estimated coefficients in Table 1 imply that an increase of one standard deviation in devaluation sensitivity doubles the observable effect of model-based learning, whereas if devaluation sensitivity is one standard deviation below the mean, it eliminates model-based learning altogether.Fig. 3

Bottom Line: Studies in humans and rodents have suggested that behavior can at times be "goal-directed"-that is, planned, and purposeful-and at times "habitual"-that is, inflexible and automatically evoked by stimuli.We then tested for habits by devaluing one of the rewards that had reinforced behavior.In each case, we found that individual differences in model-based learning predicted the participants' subsequent sensitivity to outcome devaluation, suggesting that an associative mechanism underlies a bias toward habit formation in healthy individuals.

View Article: PubMed Central - PubMed

Affiliation: Department of Psychology, New York University, 6 Washington Place, New York, NY, 10003, USA, claire.gillan@gmail.com.

ABSTRACT
Studies in humans and rodents have suggested that behavior can at times be "goal-directed"-that is, planned, and purposeful-and at times "habitual"-that is, inflexible and automatically evoked by stimuli. This distinction is central to conceptions of pathological compulsion, as in drug abuse and obsessive-compulsive disorder. Evidence for the distinction has primarily come from outcome devaluation studies, in which the sensitivity of a previously learned behavior to motivational change is used to assay the dominance of habits versus goal-directed actions. However, little is known about how habits and goal-directed control arise. Specifically, in the present study we sought to reveal the trial-by-trial dynamics of instrumental learning that would promote, and protect against, developing habits. In two complementary experiments with independent samples, participants completed a sequential decision task that dissociated two computational-learning mechanisms, model-based and model-free. We then tested for habits by devaluing one of the rewards that had reinforced behavior. In each case, we found that individual differences in model-based learning predicted the participants' subsequent sensitivity to outcome devaluation, suggesting that an associative mechanism underlies a bias toward habit formation in healthy individuals.

No MeSH data available.


Related in: MedlinePlus