Limits...
Learning the microstructure of successful behavior.

Charlesworth JD, Tumer EC, Warren TL, Brainard MS - Nat. Neurosci. (2011)

Bottom Line: However, successful performance of many motor skills, such as speech articulation, also requires learning behavioral trajectories that vary continuously over time.A simple principle predicted the detailed structure of learning: birds learned to produce the average of the behavioral trajectories associated with successful outcomes.This learning rule accurately predicted the structure of learning at a millisecond timescale, demonstrating that the nervous system records fine-grained details of successful behavior and uses this information to guide learning.

View Article: PubMed Central - PubMed

Affiliation: W M Keck Center for Integrative Neuroscience, University of California, San Francisco, California, USA. jcharles@phy.ucsf.edu

ABSTRACT
Reinforcement signals indicating success or failure are known to alter the probability of selecting between distinct actions. However, successful performance of many motor skills, such as speech articulation, also requires learning behavioral trajectories that vary continuously over time. Here, we investigated how temporally discrete reinforcement signals shape a continuous behavioral trajectory, the fundamental frequency of adult Bengalese finch song. We provided reinforcement contingent on fundamental frequency performance only at one point in the song. Learned changes to fundamental frequency were maximal at this point, but also extended both earlier and later in the fundamental frequency trajectory. A simple principle predicted the detailed structure of learning: birds learned to produce the average of the behavioral trajectories associated with successful outcomes. This learning rule accurately predicted the structure of learning at a millisecond timescale, demonstrating that the nervous system records fine-grained details of successful behavior and uses this information to guide learning.

Show MeSH
The range of variation constrains learninga. Summary data for experiments (n=5) that consisted of a baseline period, followed by a 100% aversive reinforcement period, followed by an instructive aversive reinforcement period. Circles and vertical lines indicate mean ± s.e.m. of FF for an entire day. The baseline period was used to characterize the natural range of within-day and between-day FF variation. In the 100% period, all syllables received white noise. In the instructive period, all syllables with high FF were allowed to escape white noise. Days 7 and 11 correspond to the final day in the 100% and instructive period, respectively. b. Comparison of within-day FF variation. The CV of FF was calculated for each of the baseline and 100% days shown in A. c. Comparison of between-days FF variation. Change in mean FF between days was computed for each of the pairs of days shown. In a–c, vertical lines denote ± s.e.m.
© Copyright Policy
Related In: Results  -  Collection


getmorefigures.php?uid=PMC3045469&req=5

Figure 6: The range of variation constrains learninga. Summary data for experiments (n=5) that consisted of a baseline period, followed by a 100% aversive reinforcement period, followed by an instructive aversive reinforcement period. Circles and vertical lines indicate mean ± s.e.m. of FF for an entire day. The baseline period was used to characterize the natural range of within-day and between-day FF variation. In the 100% period, all syllables received white noise. In the instructive period, all syllables with high FF were allowed to escape white noise. Days 7 and 11 correspond to the final day in the 100% and instructive period, respectively. b. Comparison of within-day FF variation. The CV of FF was calculated for each of the baseline and 100% days shown in A. c. Comparison of between-days FF variation. Change in mean FF between days was computed for each of the pairs of days shown. In a–c, vertical lines denote ± s.e.m.

Mentions: Fundamental frequency performance was stable throughout the 100% period, whereas rapid and robust learning occurred during the instructive period (Fig. 6a, n=5). Furthermore, we found no evidence for an increase in trial-by-trial (Fig. 6b) or day-to-day (Fig. 6c) fundamental frequency variation during the 100% period. In summary, we saw no change in behavior when all variants of the behavior were unsuccessful. Our results demonstrate that the natural pattern of behavioral variation constrains learning in the presence of a fixed reinforcement contingency.


Learning the microstructure of successful behavior.

Charlesworth JD, Tumer EC, Warren TL, Brainard MS - Nat. Neurosci. (2011)

The range of variation constrains learninga. Summary data for experiments (n=5) that consisted of a baseline period, followed by a 100% aversive reinforcement period, followed by an instructive aversive reinforcement period. Circles and vertical lines indicate mean ± s.e.m. of FF for an entire day. The baseline period was used to characterize the natural range of within-day and between-day FF variation. In the 100% period, all syllables received white noise. In the instructive period, all syllables with high FF were allowed to escape white noise. Days 7 and 11 correspond to the final day in the 100% and instructive period, respectively. b. Comparison of within-day FF variation. The CV of FF was calculated for each of the baseline and 100% days shown in A. c. Comparison of between-days FF variation. Change in mean FF between days was computed for each of the pairs of days shown. In a–c, vertical lines denote ± s.e.m.
© Copyright Policy
Related In: Results  -  Collection

Show All Figures
getmorefigures.php?uid=PMC3045469&req=5

Figure 6: The range of variation constrains learninga. Summary data for experiments (n=5) that consisted of a baseline period, followed by a 100% aversive reinforcement period, followed by an instructive aversive reinforcement period. Circles and vertical lines indicate mean ± s.e.m. of FF for an entire day. The baseline period was used to characterize the natural range of within-day and between-day FF variation. In the 100% period, all syllables received white noise. In the instructive period, all syllables with high FF were allowed to escape white noise. Days 7 and 11 correspond to the final day in the 100% and instructive period, respectively. b. Comparison of within-day FF variation. The CV of FF was calculated for each of the baseline and 100% days shown in A. c. Comparison of between-days FF variation. Change in mean FF between days was computed for each of the pairs of days shown. In a–c, vertical lines denote ± s.e.m.
Mentions: Fundamental frequency performance was stable throughout the 100% period, whereas rapid and robust learning occurred during the instructive period (Fig. 6a, n=5). Furthermore, we found no evidence for an increase in trial-by-trial (Fig. 6b) or day-to-day (Fig. 6c) fundamental frequency variation during the 100% period. In summary, we saw no change in behavior when all variants of the behavior were unsuccessful. Our results demonstrate that the natural pattern of behavioral variation constrains learning in the presence of a fixed reinforcement contingency.

Bottom Line: However, successful performance of many motor skills, such as speech articulation, also requires learning behavioral trajectories that vary continuously over time.A simple principle predicted the detailed structure of learning: birds learned to produce the average of the behavioral trajectories associated with successful outcomes.This learning rule accurately predicted the structure of learning at a millisecond timescale, demonstrating that the nervous system records fine-grained details of successful behavior and uses this information to guide learning.

View Article: PubMed Central - PubMed

Affiliation: W M Keck Center for Integrative Neuroscience, University of California, San Francisco, California, USA. jcharles@phy.ucsf.edu

ABSTRACT
Reinforcement signals indicating success or failure are known to alter the probability of selecting between distinct actions. However, successful performance of many motor skills, such as speech articulation, also requires learning behavioral trajectories that vary continuously over time. Here, we investigated how temporally discrete reinforcement signals shape a continuous behavioral trajectory, the fundamental frequency of adult Bengalese finch song. We provided reinforcement contingent on fundamental frequency performance only at one point in the song. Learned changes to fundamental frequency were maximal at this point, but also extended both earlier and later in the fundamental frequency trajectory. A simple principle predicted the detailed structure of learning: birds learned to produce the average of the behavioral trajectories associated with successful outcomes. This learning rule accurately predicted the structure of learning at a millisecond timescale, demonstrating that the nervous system records fine-grained details of successful behavior and uses this information to guide learning.

Show MeSH