Limits...
Structural drift: the population dynamics of sequential learning.

Crutchfield JP, Whalen S - PLoS Comput. Biol. (2012)

Bottom Line: We introduce a theory of sequential causal inference in which learners in a chain estimate a structural model from their upstream "teacher" and then pass samples from the model to their downstream "student".It extends the population dynamics of genetic drift, recasting Kimura's selectively neutral theory as a special case of a generalized drift process using structured populations with memory.We also demonstrate how the organization of drift process space controls fidelity, facilitates innovations, and leads to information loss in sequential learning with and without memory.

View Article: PubMed Central - PubMed

Affiliation: Complexity Sciences Center, Physics Department, University of California Davis, Davis, California, United States of America. chaos@ucdavis.edu

ABSTRACT
We introduce a theory of sequential causal inference in which learners in a chain estimate a structural model from their upstream "teacher" and then pass samples from the model to their downstream "student". It extends the population dynamics of genetic drift, recasting Kimura's selectively neutral theory as a special case of a generalized drift process using structured populations with memory. We examine the diffusion and fixation properties of several drift processes and propose applications to learning, inference, and evolution. We also demonstrate how the organization of drift process space controls fidelity, facilitates innovations, and leads to information loss in sequential learning with and without memory.

Show MeSH
Complexity-entropy diagram for  realizations of the Golden Mean Process with , both without (left) and with (right) structural innovation.Alternating Process and Fixed Coin pathways are clearly visible in the left panel where the Golden Mean subspace exists on the upper curve and the Biased Coin subspace exists on the line . -Machines within the same isostructural subspace have identical colors.
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC3369870&req=5

pcbi-1002510-g010: Complexity-entropy diagram for realizations of the Golden Mean Process with , both without (left) and with (right) structural innovation.Alternating Process and Fixed Coin pathways are clearly visible in the left panel where the Golden Mean subspace exists on the upper curve and the Biased Coin subspace exists on the line . -Machines within the same isostructural subspace have identical colors.

Mentions: Two such CE diagrams are shown in Figure 10, illustrating different subspaces and stasis points reachable by the Golden Mean Process during structural drift. Consider the left panel first. An -machine reaches stasis by transforming into either the Fixed Coin or the Alternating Process. To reach the former, the -machine begins on the upper curve in the left panel and drifts until the transition probability nears zero and the inference algorithm decides to merge states in the next generation. This forces the -machine to jump to the Biased Coin subspace on the line where it will most likely diffuse until the Fixed Coin stasis point at is reached. If instead the transition probability drifts towards zero, the Golden Mean stays on the upper curve until reaching the Alternating Process stasis point at . Thus, the two stasis points are differentiated not by but by , with the Alternating Process requiring 1 bit of memory to track its internal state and the Biased Coin Process requiring none.


Structural drift: the population dynamics of sequential learning.

Crutchfield JP, Whalen S - PLoS Comput. Biol. (2012)

Complexity-entropy diagram for  realizations of the Golden Mean Process with , both without (left) and with (right) structural innovation.Alternating Process and Fixed Coin pathways are clearly visible in the left panel where the Golden Mean subspace exists on the upper curve and the Biased Coin subspace exists on the line . -Machines within the same isostructural subspace have identical colors.
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC3369870&req=5

pcbi-1002510-g010: Complexity-entropy diagram for realizations of the Golden Mean Process with , both without (left) and with (right) structural innovation.Alternating Process and Fixed Coin pathways are clearly visible in the left panel where the Golden Mean subspace exists on the upper curve and the Biased Coin subspace exists on the line . -Machines within the same isostructural subspace have identical colors.
Mentions: Two such CE diagrams are shown in Figure 10, illustrating different subspaces and stasis points reachable by the Golden Mean Process during structural drift. Consider the left panel first. An -machine reaches stasis by transforming into either the Fixed Coin or the Alternating Process. To reach the former, the -machine begins on the upper curve in the left panel and drifts until the transition probability nears zero and the inference algorithm decides to merge states in the next generation. This forces the -machine to jump to the Biased Coin subspace on the line where it will most likely diffuse until the Fixed Coin stasis point at is reached. If instead the transition probability drifts towards zero, the Golden Mean stays on the upper curve until reaching the Alternating Process stasis point at . Thus, the two stasis points are differentiated not by but by , with the Alternating Process requiring 1 bit of memory to track its internal state and the Biased Coin Process requiring none.

Bottom Line: We introduce a theory of sequential causal inference in which learners in a chain estimate a structural model from their upstream "teacher" and then pass samples from the model to their downstream "student".It extends the population dynamics of genetic drift, recasting Kimura's selectively neutral theory as a special case of a generalized drift process using structured populations with memory.We also demonstrate how the organization of drift process space controls fidelity, facilitates innovations, and leads to information loss in sequential learning with and without memory.

View Article: PubMed Central - PubMed

Affiliation: Complexity Sciences Center, Physics Department, University of California Davis, Davis, California, United States of America. chaos@ucdavis.edu

ABSTRACT
We introduce a theory of sequential causal inference in which learners in a chain estimate a structural model from their upstream "teacher" and then pass samples from the model to their downstream "student". It extends the population dynamics of genetic drift, recasting Kimura's selectively neutral theory as a special case of a generalized drift process using structured populations with memory. We examine the diffusion and fixation properties of several drift processes and propose applications to learning, inference, and evolution. We also demonstrate how the organization of drift process space controls fidelity, facilitates innovations, and leads to information loss in sequential learning with and without memory.

Show MeSH