Limits...
A dual role for prediction error in associative learning.

den Ouden HE, Friston KJ, Daw ND, McIntosh AR, Stephan KE - Cereb. Cortex (2008)

Bottom Line: We investigated the underlying mechanism by embedding the RW model into a DCM to show that auditory to visual connectivity changed significantly over time as a function of prediction error.Thus, consistent with predictive coding models of perception, associative learning is mediated by prediction-error dependent changes in connectivity.These results posit a dual role for prediction-error in encoding surprise and driving associative plasticity.

View Article: PubMed Central - PubMed

Affiliation: Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, 12 Queen Square, London WC1N 3BG, UK. h.denouden@fil.ion.ucl.ac.uk

ABSTRACT
Confronted with a rich sensory environment, the brain must learn statistical regularities across sensory domains to construct causal models of the world. Here, we used functional magnetic resonance imaging and dynamic causal modeling (DCM) to furnish neurophysiological evidence that statistical associations are learnt, even when task-irrelevant. Subjects performed an audio-visual target-detection task while being exposed to distractor stimuli. Unknown to them, auditory distractors predicted the presence or absence of subsequent visual distractors. We modeled incidental learning of these associations using a Rescorla-Wagner (RW) model. Activity in primary visual cortex and putamen reflected learning-dependent surprise: these areas responded progressively more to unpredicted, and progressively less to predicted visual stimuli. Critically, this prediction-error response was observed even when the absence of a visual stimulus was surprising. We investigated the underlying mechanism by embedding the RW model into a DCM to show that auditory to visual connectivity changed significantly over time as a function of prediction error. Thus, consistent with predictive coding models of perception, associative learning is mediated by prediction-error dependent changes in connectivity. These results posit a dual role for prediction-error in encoding surprise and driving associative plasticity.

Show MeSH

Related in: MedlinePlus

fMRI results. (A) Significant activations in V1 as a function of RW learning, for both the 4-way interaction (CS type × CS presence × visual outcome × RW learning; red), and the simple (3-way) interaction (blue), which is restricted to the CS+ trials (x = −6, also showing the caudate activation) and (B) in the putamen bilaterally (y = 6), displayed on the mean structural image across all subjects. (C) z = 12. Significant 3-way interaction CS type × CS presence × RW learning in the DLPFC and left putamen (red). This interaction is driven by the CS+ trials, as shown by the simple interaction in blue.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC2665159&req=5

fig5: fMRI results. (A) Significant activations in V1 as a function of RW learning, for both the 4-way interaction (CS type × CS presence × visual outcome × RW learning; red), and the simple (3-way) interaction (blue), which is restricted to the CS+ trials (x = −6, also showing the caudate activation) and (B) in the putamen bilaterally (y = 6), displayed on the mean structural image across all subjects. (C) z = 12. Significant 3-way interaction CS type × CS presence × RW learning in the DLPFC and left putamen (red). This interaction is driven by the CS+ trials, as shown by the simple interaction in blue.

Mentions: The question addressed by DCM was whether learning effects in V1 could be explained by changes in the connectivity of a simple auditory–visual network. Our DCMs modeled the entire time series, so data from all trials or conditions, trying to explain regional activations by condition-dependent changes in connectivity. We tested 3 simple models that could potentially account for the interaction we found in V1. These models were fitted separately to each subject's data and compared using Bayesian model selection (Penny et al. 2004). In these models, auditory and visual stimuli from all trials elicited activity directly in their respective primary sensory areas (see Fig. 4). These driving inputs were modeled as individual events. The first model only had a connection from A1 to V1, whereas the second and third models included the reciprocal connection (see Fig. 5). The A1 → V1 connection in model 1 and 2, and the V1 → A1 connection in model 3 were modulated by the Hadamard product (point-wise multiplication) of the RW associative strength and a vector encoding visual outcome (1 for visual stimulus present, −1 for visual stimulus absent) during CS+ trials. In the first 2 models, this modulatory effect corresponds to the interaction of the auditory CS+ prediction with the visual outcome and models a learning-dependent contribution from CS+ responses in auditory cortex to visual cortex responses that depends on whether the visual stimulus was present or not (c.f., a prediction error that rests on top-down signals from auditory areas). In the third model, which represented a control suggested by one of our reviewers, this modulatory effect acted on the reverse connection, V1→A1.


A dual role for prediction error in associative learning.

den Ouden HE, Friston KJ, Daw ND, McIntosh AR, Stephan KE - Cereb. Cortex (2008)

fMRI results. (A) Significant activations in V1 as a function of RW learning, for both the 4-way interaction (CS type × CS presence × visual outcome × RW learning; red), and the simple (3-way) interaction (blue), which is restricted to the CS+ trials (x = −6, also showing the caudate activation) and (B) in the putamen bilaterally (y = 6), displayed on the mean structural image across all subjects. (C) z = 12. Significant 3-way interaction CS type × CS presence × RW learning in the DLPFC and left putamen (red). This interaction is driven by the CS+ trials, as shown by the simple interaction in blue.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC2665159&req=5

fig5: fMRI results. (A) Significant activations in V1 as a function of RW learning, for both the 4-way interaction (CS type × CS presence × visual outcome × RW learning; red), and the simple (3-way) interaction (blue), which is restricted to the CS+ trials (x = −6, also showing the caudate activation) and (B) in the putamen bilaterally (y = 6), displayed on the mean structural image across all subjects. (C) z = 12. Significant 3-way interaction CS type × CS presence × RW learning in the DLPFC and left putamen (red). This interaction is driven by the CS+ trials, as shown by the simple interaction in blue.
Mentions: The question addressed by DCM was whether learning effects in V1 could be explained by changes in the connectivity of a simple auditory–visual network. Our DCMs modeled the entire time series, so data from all trials or conditions, trying to explain regional activations by condition-dependent changes in connectivity. We tested 3 simple models that could potentially account for the interaction we found in V1. These models were fitted separately to each subject's data and compared using Bayesian model selection (Penny et al. 2004). In these models, auditory and visual stimuli from all trials elicited activity directly in their respective primary sensory areas (see Fig. 4). These driving inputs were modeled as individual events. The first model only had a connection from A1 to V1, whereas the second and third models included the reciprocal connection (see Fig. 5). The A1 → V1 connection in model 1 and 2, and the V1 → A1 connection in model 3 were modulated by the Hadamard product (point-wise multiplication) of the RW associative strength and a vector encoding visual outcome (1 for visual stimulus present, −1 for visual stimulus absent) during CS+ trials. In the first 2 models, this modulatory effect corresponds to the interaction of the auditory CS+ prediction with the visual outcome and models a learning-dependent contribution from CS+ responses in auditory cortex to visual cortex responses that depends on whether the visual stimulus was present or not (c.f., a prediction error that rests on top-down signals from auditory areas). In the third model, which represented a control suggested by one of our reviewers, this modulatory effect acted on the reverse connection, V1→A1.

Bottom Line: We investigated the underlying mechanism by embedding the RW model into a DCM to show that auditory to visual connectivity changed significantly over time as a function of prediction error.Thus, consistent with predictive coding models of perception, associative learning is mediated by prediction-error dependent changes in connectivity.These results posit a dual role for prediction-error in encoding surprise and driving associative plasticity.

View Article: PubMed Central - PubMed

Affiliation: Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, 12 Queen Square, London WC1N 3BG, UK. h.denouden@fil.ion.ucl.ac.uk

ABSTRACT
Confronted with a rich sensory environment, the brain must learn statistical regularities across sensory domains to construct causal models of the world. Here, we used functional magnetic resonance imaging and dynamic causal modeling (DCM) to furnish neurophysiological evidence that statistical associations are learnt, even when task-irrelevant. Subjects performed an audio-visual target-detection task while being exposed to distractor stimuli. Unknown to them, auditory distractors predicted the presence or absence of subsequent visual distractors. We modeled incidental learning of these associations using a Rescorla-Wagner (RW) model. Activity in primary visual cortex and putamen reflected learning-dependent surprise: these areas responded progressively more to unpredicted, and progressively less to predicted visual stimuli. Critically, this prediction-error response was observed even when the absence of a visual stimulus was surprising. We investigated the underlying mechanism by embedding the RW model into a DCM to show that auditory to visual connectivity changed significantly over time as a function of prediction error. Thus, consistent with predictive coding models of perception, associative learning is mediated by prediction-error dependent changes in connectivity. These results posit a dual role for prediction-error in encoding surprise and driving associative plasticity.

Show MeSH
Related in: MedlinePlus