Limits...
Emergence of functional hierarchy in a multiple timescale neural network model: a humanoid robot experiment.

Yamashita Y, Tani J - PLoS Comput. Biol. (2008)

Bottom Line: The current model neither makes use of separate local modules to represent primitives nor introduces explicit hierarchical structure.In experiments, the proposed network model, coordinating the physical body of a humanoid robot through high-dimensional sensori-motor control, also successfully situated itself within a physical environment.Our results suggest that it is not only the spatial connections between neurons but also the timescales of neural activity that act as important mechanisms leading to functional hierarchy in neural systems.

View Article: PubMed Central - PubMed

Affiliation: Laboratory for Behavior and Dynamic Cognition, RIKEN Brain Science Institute, Wako-shi, Saitama, Japan. yamay@brain.riken.jp

ABSTRACT
It is generally thought that skilled behavior in human beings results from a functional hierarchy of the motor control system, within which reusable motor primitives are flexibly integrated into various sensori-motor sequence patterns. The underlying neural mechanisms governing the way in which continuous sensori-motor flows are segmented into primitives and the way in which series of primitives are integrated into various behavior sequences have, however, not yet been clarified. In earlier studies, this functional hierarchy has been realized through the use of explicit hierarchical structure, with local modules representing motor primitives in the lower level and a higher module representing sequences of primitives switched via additional mechanisms such as gate-selecting. When sequences contain similarities and overlap, however, a conflict arises in such earlier models between generalization and segmentation, induced by this separated modular structure. To address this issue, we propose a different type of neural network model. The current model neither makes use of separate local modules to represent primitives nor introduces explicit hierarchical structure. Rather than forcing architectural hierarchy onto the system, functional hierarchy emerges through a form of self-organization that is based on two distinct types of neurons, each with different time properties ("multiple timescales"). Through the introduction of multiple timescales, continuous sequences of behavior are segmented into reusable primitives, and the primitives, in turn, are flexibly integrated into novel sequences. In experiments, the proposed network model, coordinating the physical body of a humanoid robot through high-dimensional sensori-motor control, also successfully situated itself within a physical environment. Our results suggest that it is not only the spatial connections between neurons but also the timescales of neural activity that act as important mechanisms leading to functional hierarchy in neural systems.

Show MeSH

Related in: MedlinePlus

System details.The main part of the system is the CTRNN. The total number of CTRNN unitswas 180. The first 100 units (indicesi = 1‥100)correspond to input-output units (O). Among inputunits, the first 64 units (indicesi = 1‥64)correspond to proprioceptive inputs (M), whereas thelast 36 units (indicesi = 65‥100)correspond to vision inputs (S). The remaining 80 units(indicesi = 101‥180)correspond to the context units. Among the context units, the first 60units (indicesi = 101‥160)correspond to the fast context units (Cf), and the last20 units (indicesi = 161‥180)correspond to the slow context units (Cs). Inputs tothe system were the proprioception mˆt and the vision sense ŝt, which were transformed into sparsely encoded vectors usingtopology preserving maps (TPM, Equation 3), one map corresponding toproprioception (TPMm) and one map corresponding to vision (TPMs). A100-dimensional vector, transformed by the TPM(pi,t) and previous activation levels of thecontext units yi,t−1,is set to the neural states xi,t (Equation7). Membrane potential (ui,t) and activation(yi,t) of each unit are calculatedusing Equation 5 and Equation 6, respectively. Outputs of the CTRNN(yi,t,i∈O) are transformed into10 dimensional vectors(mt+1 andst+1) usinginverse computation of the TPM (iTPM, Equation 4). These 10 dimensionalvectors correspond to predictions of the proprioceptionmt+1 and the visionsense st+1 for the nexttime step. This prediction of the proprioceptionmt+1 was sent to therobot in the form of target joint angles, which acted as motor commandsfor the robot in generating movements and interacting with the physicalenvironment. Changes in the environment resulting from this interactionwere sent back to the system in the form of sensory feedback. Intraining, output of the CTRNN (yi,t,i∈O) is compared with thedesired output y*i,t calculated from target sensori-motor statesm*t+1and s*t+1, usingthe same TPMs.
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC2570613&req=5

pcbi-1000220-g010: System details.The main part of the system is the CTRNN. The total number of CTRNN unitswas 180. The first 100 units (indicesi = 1‥100)correspond to input-output units (O). Among inputunits, the first 64 units (indicesi = 1‥64)correspond to proprioceptive inputs (M), whereas thelast 36 units (indicesi = 65‥100)correspond to vision inputs (S). The remaining 80 units(indicesi = 101‥180)correspond to the context units. Among the context units, the first 60units (indicesi = 101‥160)correspond to the fast context units (Cf), and the last20 units (indicesi = 161‥180)correspond to the slow context units (Cs). Inputs tothe system were the proprioception mˆt and the vision sense ŝt, which were transformed into sparsely encoded vectors usingtopology preserving maps (TPM, Equation 3), one map corresponding toproprioception (TPMm) and one map corresponding to vision (TPMs). A100-dimensional vector, transformed by the TPM(pi,t) and previous activation levels of thecontext units yi,t−1,is set to the neural states xi,t (Equation7). Membrane potential (ui,t) and activation(yi,t) of each unit are calculatedusing Equation 5 and Equation 6, respectively. Outputs of the CTRNN(yi,t,i∈O) are transformed into10 dimensional vectors(mt+1 andst+1) usinginverse computation of the TPM (iTPM, Equation 4). These 10 dimensionalvectors correspond to predictions of the proprioceptionmt+1 and the visionsense st+1 for the nexttime step. This prediction of the proprioceptionmt+1 was sent to therobot in the form of target joint angles, which acted as motor commandsfor the robot in generating movements and interacting with the physicalenvironment. Changes in the environment resulting from this interactionwere sent back to the system in the form of sensory feedback. Intraining, output of the CTRNN (yi,t,i∈O) is compared with thedesired output y*i,t calculated from target sensori-motor statesm*t+1and s*t+1, usingthe same TPMs.

Mentions: Inputs to the system were sparsely encoded in the form of a population codingusing conventional topology preserving maps (TPM, [67]), one mapcorresponding to proprioception and one map corresponding to vision (Figure 10). The TPM is a typeof a neural network that produces a discretized representation of the inputspace of training samples. The characteristic feature of the TPM is that itpreserves topological properties of the input space. This sparse encoding ofsensori-motor trajectories reduces the overlaps of sensori-motor sequences. Thesize of the TPMs were 64 (8×8) for proprioception and 36(6×6) for vision sense, respectively. 10 dimensional proprioceptiveand visual inputs were thus transformed into 100 dimensional sparsely encodedvectors.


Emergence of functional hierarchy in a multiple timescale neural network model: a humanoid robot experiment.

Yamashita Y, Tani J - PLoS Comput. Biol. (2008)

System details.The main part of the system is the CTRNN. The total number of CTRNN unitswas 180. The first 100 units (indicesi = 1‥100)correspond to input-output units (O). Among inputunits, the first 64 units (indicesi = 1‥64)correspond to proprioceptive inputs (M), whereas thelast 36 units (indicesi = 65‥100)correspond to vision inputs (S). The remaining 80 units(indicesi = 101‥180)correspond to the context units. Among the context units, the first 60units (indicesi = 101‥160)correspond to the fast context units (Cf), and the last20 units (indicesi = 161‥180)correspond to the slow context units (Cs). Inputs tothe system were the proprioception mˆt and the vision sense ŝt, which were transformed into sparsely encoded vectors usingtopology preserving maps (TPM, Equation 3), one map corresponding toproprioception (TPMm) and one map corresponding to vision (TPMs). A100-dimensional vector, transformed by the TPM(pi,t) and previous activation levels of thecontext units yi,t−1,is set to the neural states xi,t (Equation7). Membrane potential (ui,t) and activation(yi,t) of each unit are calculatedusing Equation 5 and Equation 6, respectively. Outputs of the CTRNN(yi,t,i∈O) are transformed into10 dimensional vectors(mt+1 andst+1) usinginverse computation of the TPM (iTPM, Equation 4). These 10 dimensionalvectors correspond to predictions of the proprioceptionmt+1 and the visionsense st+1 for the nexttime step. This prediction of the proprioceptionmt+1 was sent to therobot in the form of target joint angles, which acted as motor commandsfor the robot in generating movements and interacting with the physicalenvironment. Changes in the environment resulting from this interactionwere sent back to the system in the form of sensory feedback. Intraining, output of the CTRNN (yi,t,i∈O) is compared with thedesired output y*i,t calculated from target sensori-motor statesm*t+1and s*t+1, usingthe same TPMs.
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC2570613&req=5

pcbi-1000220-g010: System details.The main part of the system is the CTRNN. The total number of CTRNN unitswas 180. The first 100 units (indicesi = 1‥100)correspond to input-output units (O). Among inputunits, the first 64 units (indicesi = 1‥64)correspond to proprioceptive inputs (M), whereas thelast 36 units (indicesi = 65‥100)correspond to vision inputs (S). The remaining 80 units(indicesi = 101‥180)correspond to the context units. Among the context units, the first 60units (indicesi = 101‥160)correspond to the fast context units (Cf), and the last20 units (indicesi = 161‥180)correspond to the slow context units (Cs). Inputs tothe system were the proprioception mˆt and the vision sense ŝt, which were transformed into sparsely encoded vectors usingtopology preserving maps (TPM, Equation 3), one map corresponding toproprioception (TPMm) and one map corresponding to vision (TPMs). A100-dimensional vector, transformed by the TPM(pi,t) and previous activation levels of thecontext units yi,t−1,is set to the neural states xi,t (Equation7). Membrane potential (ui,t) and activation(yi,t) of each unit are calculatedusing Equation 5 and Equation 6, respectively. Outputs of the CTRNN(yi,t,i∈O) are transformed into10 dimensional vectors(mt+1 andst+1) usinginverse computation of the TPM (iTPM, Equation 4). These 10 dimensionalvectors correspond to predictions of the proprioceptionmt+1 and the visionsense st+1 for the nexttime step. This prediction of the proprioceptionmt+1 was sent to therobot in the form of target joint angles, which acted as motor commandsfor the robot in generating movements and interacting with the physicalenvironment. Changes in the environment resulting from this interactionwere sent back to the system in the form of sensory feedback. Intraining, output of the CTRNN (yi,t,i∈O) is compared with thedesired output y*i,t calculated from target sensori-motor statesm*t+1and s*t+1, usingthe same TPMs.
Mentions: Inputs to the system were sparsely encoded in the form of a population codingusing conventional topology preserving maps (TPM, [67]), one mapcorresponding to proprioception and one map corresponding to vision (Figure 10). The TPM is a typeof a neural network that produces a discretized representation of the inputspace of training samples. The characteristic feature of the TPM is that itpreserves topological properties of the input space. This sparse encoding ofsensori-motor trajectories reduces the overlaps of sensori-motor sequences. Thesize of the TPMs were 64 (8×8) for proprioception and 36(6×6) for vision sense, respectively. 10 dimensional proprioceptiveand visual inputs were thus transformed into 100 dimensional sparsely encodedvectors.

Bottom Line: The current model neither makes use of separate local modules to represent primitives nor introduces explicit hierarchical structure.In experiments, the proposed network model, coordinating the physical body of a humanoid robot through high-dimensional sensori-motor control, also successfully situated itself within a physical environment.Our results suggest that it is not only the spatial connections between neurons but also the timescales of neural activity that act as important mechanisms leading to functional hierarchy in neural systems.

View Article: PubMed Central - PubMed

Affiliation: Laboratory for Behavior and Dynamic Cognition, RIKEN Brain Science Institute, Wako-shi, Saitama, Japan. yamay@brain.riken.jp

ABSTRACT
It is generally thought that skilled behavior in human beings results from a functional hierarchy of the motor control system, within which reusable motor primitives are flexibly integrated into various sensori-motor sequence patterns. The underlying neural mechanisms governing the way in which continuous sensori-motor flows are segmented into primitives and the way in which series of primitives are integrated into various behavior sequences have, however, not yet been clarified. In earlier studies, this functional hierarchy has been realized through the use of explicit hierarchical structure, with local modules representing motor primitives in the lower level and a higher module representing sequences of primitives switched via additional mechanisms such as gate-selecting. When sequences contain similarities and overlap, however, a conflict arises in such earlier models between generalization and segmentation, induced by this separated modular structure. To address this issue, we propose a different type of neural network model. The current model neither makes use of separate local modules to represent primitives nor introduces explicit hierarchical structure. Rather than forcing architectural hierarchy onto the system, functional hierarchy emerges through a form of self-organization that is based on two distinct types of neurons, each with different time properties ("multiple timescales"). Through the introduction of multiple timescales, continuous sequences of behavior are segmented into reusable primitives, and the primitives, in turn, are flexibly integrated into novel sequences. In experiments, the proposed network model, coordinating the physical body of a humanoid robot through high-dimensional sensori-motor control, also successfully situated itself within a physical environment. Our results suggest that it is not only the spatial connections between neurons but also the timescales of neural activity that act as important mechanisms leading to functional hierarchy in neural systems.

Show MeSH
Related in: MedlinePlus