Limits...
Hierarchical models in the brain.

Friston K - PLoS Comput. Biol. (2008)

Bottom Line: This means that a single model and optimisation scheme can be used to invert a wide range of models.We present the model and a brief review of its inversion to disclose the relationships among, apparently, diverse generative models of empirical data.We then show that this inversion can be formulated as a simple neural network and may provide a useful metaphor for inference and learning in the brain.

View Article: PubMed Central - PubMed

Affiliation: The Wellcome Trust Centre of Neuroimaging, University College London, London, United Kingdom. k.friston@fil.ion.ucl.ac.uk

ABSTRACT
This paper describes a general model that subsumes many parametric models for continuous data. The model comprises hidden layers of state-space or dynamic causal models, arranged so that the output of one provides input to another. The ensuing hierarchy furnishes a model for many types of data, of arbitrary complexity. Special cases range from the general linear model for static data to generalised convolution models, with system noise, for nonlinear time-series analysis. Crucially, all of these models can be inverted using exactly the same scheme, namely, dynamic expectation maximization. This means that a single model and optimisation scheme can be used to invert a wide range of models. We present the model and a brief review of its inversion to disclose the relationships among, apparently, diverse generative models of empirical data. We then show that this inversion can be formulated as a simple neural network and may provide a useful metaphor for inference and learning in the brain.

Show MeSH
Example of estimation under a mixed-effects or hierarchicallinear model.The inversion was cross-validated with expectation maximization (EM),where the M-step corresponds to restricted maximum likelihood(ReML). This example used a simple two-level model that embodiesempirical shrinkage priors on the first-level parameters. Thesemodels are also known as parametric empirical Bayes (PEB) models(left). Causes were sampled from the unit normal density to generatea response, which was used to recover the causes, given theparameters. Slight differences in the hyperparameter estimates(upper right), due to a different hyperparameterisation, have littleeffect on the conditional means of the unknown causes (lower right),which are almost indistinguishable.
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC2570625&req=5

pcbi-1000211-g003: Example of estimation under a mixed-effects or hierarchicallinear model.The inversion was cross-validated with expectation maximization (EM),where the M-step corresponds to restricted maximum likelihood(ReML). This example used a simple two-level model that embodiesempirical shrinkage priors on the first-level parameters. Thesemodels are also known as parametric empirical Bayes (PEB) models(left). Causes were sampled from the unit normal density to generatea response, which was used to recover the causes, given theparameters. Slight differences in the hyperparameter estimates(upper right), due to a different hyperparameterisation, have littleeffect on the conditional means of the unknown causes (lower right),which are almost indistinguishable.

Mentions: When the model above is linear, we have the ubiquitous hierarchical linearobservation model used in Parametric Empirical Bayes (PEB;[8]) and mixed-effects analysis of covariance(ANCOVA) analyses.(40)Here the D-Step converges after a singleiteration because the linearity of this model renders the Laplace assumptionexact. In this context, the M-Step becomes a classicalrestricted maximum likelihood (ReML) estimation of thehierarchical covariance components,Σ(i)z. It isinteresting to note that the ReML objective function and thevariational energy are formally identical under this model [15],[18]. Figure 3 shows acomparative evaluation of ReML and DEM using thesame data. The estimates are similar but not identical. This is becauseDEM hyperparameterises the covariance as a linear mixtureof precisions, whereas the ReML scheme used a linear mixture ofcovariance components.


Hierarchical models in the brain.

Friston K - PLoS Comput. Biol. (2008)

Example of estimation under a mixed-effects or hierarchicallinear model.The inversion was cross-validated with expectation maximization (EM),where the M-step corresponds to restricted maximum likelihood(ReML). This example used a simple two-level model that embodiesempirical shrinkage priors on the first-level parameters. Thesemodels are also known as parametric empirical Bayes (PEB) models(left). Causes were sampled from the unit normal density to generatea response, which was used to recover the causes, given theparameters. Slight differences in the hyperparameter estimates(upper right), due to a different hyperparameterisation, have littleeffect on the conditional means of the unknown causes (lower right),which are almost indistinguishable.
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC2570625&req=5

pcbi-1000211-g003: Example of estimation under a mixed-effects or hierarchicallinear model.The inversion was cross-validated with expectation maximization (EM),where the M-step corresponds to restricted maximum likelihood(ReML). This example used a simple two-level model that embodiesempirical shrinkage priors on the first-level parameters. Thesemodels are also known as parametric empirical Bayes (PEB) models(left). Causes were sampled from the unit normal density to generatea response, which was used to recover the causes, given theparameters. Slight differences in the hyperparameter estimates(upper right), due to a different hyperparameterisation, have littleeffect on the conditional means of the unknown causes (lower right),which are almost indistinguishable.
Mentions: When the model above is linear, we have the ubiquitous hierarchical linearobservation model used in Parametric Empirical Bayes (PEB;[8]) and mixed-effects analysis of covariance(ANCOVA) analyses.(40)Here the D-Step converges after a singleiteration because the linearity of this model renders the Laplace assumptionexact. In this context, the M-Step becomes a classicalrestricted maximum likelihood (ReML) estimation of thehierarchical covariance components,Σ(i)z. It isinteresting to note that the ReML objective function and thevariational energy are formally identical under this model [15],[18]. Figure 3 shows acomparative evaluation of ReML and DEM using thesame data. The estimates are similar but not identical. This is becauseDEM hyperparameterises the covariance as a linear mixtureof precisions, whereas the ReML scheme used a linear mixture ofcovariance components.

Bottom Line: This means that a single model and optimisation scheme can be used to invert a wide range of models.We present the model and a brief review of its inversion to disclose the relationships among, apparently, diverse generative models of empirical data.We then show that this inversion can be formulated as a simple neural network and may provide a useful metaphor for inference and learning in the brain.

View Article: PubMed Central - PubMed

Affiliation: The Wellcome Trust Centre of Neuroimaging, University College London, London, United Kingdom. k.friston@fil.ion.ucl.ac.uk

ABSTRACT
This paper describes a general model that subsumes many parametric models for continuous data. The model comprises hidden layers of state-space or dynamic causal models, arranged so that the output of one provides input to another. The ensuing hierarchy furnishes a model for many types of data, of arbitrary complexity. Special cases range from the general linear model for static data to generalised convolution models, with system noise, for nonlinear time-series analysis. Crucially, all of these models can be inverted using exactly the same scheme, namely, dynamic expectation maximization. This means that a single model and optimisation scheme can be used to invert a wide range of models. We present the model and a brief review of its inversion to disclose the relationships among, apparently, diverse generative models of empirical data. We then show that this inversion can be formulated as a simple neural network and may provide a useful metaphor for inference and learning in the brain.

Show MeSH