Limits...
Discriminating Non-native Vowels on the Basis of Multimodal, Auditory or Visual Information: Effects on Infants' Looking Patterns and Discrimination.

Ter Schure S, Junge C, Boersma P - Front Psychol (2016)

Bottom Line: This study tested whether infants' phonological perception is shaped by distributions of visual speech as well as by distributions of auditory speech, by comparing learning from multimodal (i.e., auditory-visual), visual-only, or auditory-only information.We used eye tracking to measure effects of distribution and sensory modality on infants' discrimination of the contrast.We propose that by 8 months, infants' native vowel categories are established insofar that learning a novel contrast is supported by attention to additional information, such as visual articulations.

View Article: PubMed Central - PubMed

Affiliation: Linguistics, University of Amsterdam Amsterdam, Netherlands.

ABSTRACT
Infants' perception of speech sound contrasts is modulated by their language environment, for example by the statistical distributions of the speech sounds they hear. Infants learn to discriminate speech sounds better when their input contains a two-peaked frequency distribution of those speech sounds than when their input contains a one-peaked frequency distribution. Effects of frequency distributions on phonetic learning have been tested almost exclusively for auditory input. But auditory speech is usually accompanied by visual information, that is, by visible articulations. This study tested whether infants' phonological perception is shaped by distributions of visual speech as well as by distributions of auditory speech, by comparing learning from multimodal (i.e., auditory-visual), visual-only, or auditory-only information. Dutch 8-month-old infants were exposed to either a one-peaked or two-peaked distribution from a continuum of vowels that formed a contrast in English, but not in Dutch. We used eye tracking to measure effects of distribution and sensory modality on infants' discrimination of the contrast. Although there were no overall effects of distribution or modality, separate t-tests in each of the six training conditions demonstrated significant discrimination of the vowel contrast in the two-peaked multimodal condition. For the modalities where the mouth was visible (visual-only and multimodal) we further examined infant looking patterns for the dynamic speaker's face. Infants in the two-peaked multimodal condition looked longer at her mouth than infants in any of the three other conditions. We propose that by 8 months, infants' native vowel categories are established insofar that learning a novel contrast is supported by attention to additional information, such as visual articulations.

No MeSH data available.


Related in: MedlinePlus

Infants’ fixations to the speaker’s mouth (left) and eyes (right) calculated as proportions to total face gaze, for each of the two blocks of the training phase. Solid lines vs. dashed lines reflect 1-peaked vs. 2-peaked trainings; color differentiates between multimodal (blue) and visual (red) trainings. Error bars denote one standard error of the mean.
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4836047&req=5

Figure 4: Infants’ fixations to the speaker’s mouth (left) and eyes (right) calculated as proportions to total face gaze, for each of the two blocks of the training phase. Solid lines vs. dashed lines reflect 1-peaked vs. 2-peaked trainings; color differentiates between multimodal (blue) and visual (red) trainings. Error bars denote one standard error of the mean.

Mentions: For the mouth region, there was a marginal effect of training block [F(1,60) = 3.61, p = 0.062], which did not interact with any between-subject factors [all F(1,60) < 1.4; all p > 0.25]: overall, infants slightly decreased their looks to the mouth area in the second block. Irrespective of the course of the training, however, we observed a main effect of Distribution [F(1,60) = 5.29, p = 0.025] and an interaction between Modality and Distribution [F(1,60) = 5.01; p = 0.029]; the main effect of Modality was insignificant [F(1,60) = 1.92; p = 0.171]. The main effect of Distribution indicated that infants in the two-peaked conditions fixated the mouth area more often than infants in the one-peaked conditions, but this main effect was mainly driven by looking performance in the two-peaked multimodal group, as Figure 4 shows: across training, it was the two-peaked multimodal condition in which infants looked 7 to 9 percent more to the mouth than in the other three conditions. Post hoc analyses (with α set to 0.01673) show that throughout training, the two-peaked multimodal group scanned the mouth more often than the one-peaked multimodal (mean difference 8.6%, 98.333% CI = 2.4 ∼ 14.8%, p = 0.0011) and more than the one-peaked or two-peaked visual groups (mean difference 7.0%, 98.333% CI = 0.4 ∼ 13.6%, p = 0.012; mean difference 6.9%, 98.333% CI = 0.3 ∼ 13.5%, p = 0.013, respectively). Thus, although we do not see the expected main effect of Modality, there is a significant interaction between Modality and Distribution on infants’ mouth fixations during training. This interaction highlights that Modality can affect infants’ looking behavior to the mouth, albeit in an indirect fashion, that is, it is dependent on the type of distribution infants received.


Discriminating Non-native Vowels on the Basis of Multimodal, Auditory or Visual Information: Effects on Infants' Looking Patterns and Discrimination.

Ter Schure S, Junge C, Boersma P - Front Psychol (2016)

Infants’ fixations to the speaker’s mouth (left) and eyes (right) calculated as proportions to total face gaze, for each of the two blocks of the training phase. Solid lines vs. dashed lines reflect 1-peaked vs. 2-peaked trainings; color differentiates between multimodal (blue) and visual (red) trainings. Error bars denote one standard error of the mean.
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4836047&req=5

Figure 4: Infants’ fixations to the speaker’s mouth (left) and eyes (right) calculated as proportions to total face gaze, for each of the two blocks of the training phase. Solid lines vs. dashed lines reflect 1-peaked vs. 2-peaked trainings; color differentiates between multimodal (blue) and visual (red) trainings. Error bars denote one standard error of the mean.
Mentions: For the mouth region, there was a marginal effect of training block [F(1,60) = 3.61, p = 0.062], which did not interact with any between-subject factors [all F(1,60) < 1.4; all p > 0.25]: overall, infants slightly decreased their looks to the mouth area in the second block. Irrespective of the course of the training, however, we observed a main effect of Distribution [F(1,60) = 5.29, p = 0.025] and an interaction between Modality and Distribution [F(1,60) = 5.01; p = 0.029]; the main effect of Modality was insignificant [F(1,60) = 1.92; p = 0.171]. The main effect of Distribution indicated that infants in the two-peaked conditions fixated the mouth area more often than infants in the one-peaked conditions, but this main effect was mainly driven by looking performance in the two-peaked multimodal group, as Figure 4 shows: across training, it was the two-peaked multimodal condition in which infants looked 7 to 9 percent more to the mouth than in the other three conditions. Post hoc analyses (with α set to 0.01673) show that throughout training, the two-peaked multimodal group scanned the mouth more often than the one-peaked multimodal (mean difference 8.6%, 98.333% CI = 2.4 ∼ 14.8%, p = 0.0011) and more than the one-peaked or two-peaked visual groups (mean difference 7.0%, 98.333% CI = 0.4 ∼ 13.6%, p = 0.012; mean difference 6.9%, 98.333% CI = 0.3 ∼ 13.5%, p = 0.013, respectively). Thus, although we do not see the expected main effect of Modality, there is a significant interaction between Modality and Distribution on infants’ mouth fixations during training. This interaction highlights that Modality can affect infants’ looking behavior to the mouth, albeit in an indirect fashion, that is, it is dependent on the type of distribution infants received.

Bottom Line: This study tested whether infants' phonological perception is shaped by distributions of visual speech as well as by distributions of auditory speech, by comparing learning from multimodal (i.e., auditory-visual), visual-only, or auditory-only information.We used eye tracking to measure effects of distribution and sensory modality on infants' discrimination of the contrast.We propose that by 8 months, infants' native vowel categories are established insofar that learning a novel contrast is supported by attention to additional information, such as visual articulations.

View Article: PubMed Central - PubMed

Affiliation: Linguistics, University of Amsterdam Amsterdam, Netherlands.

ABSTRACT
Infants' perception of speech sound contrasts is modulated by their language environment, for example by the statistical distributions of the speech sounds they hear. Infants learn to discriminate speech sounds better when their input contains a two-peaked frequency distribution of those speech sounds than when their input contains a one-peaked frequency distribution. Effects of frequency distributions on phonetic learning have been tested almost exclusively for auditory input. But auditory speech is usually accompanied by visual information, that is, by visible articulations. This study tested whether infants' phonological perception is shaped by distributions of visual speech as well as by distributions of auditory speech, by comparing learning from multimodal (i.e., auditory-visual), visual-only, or auditory-only information. Dutch 8-month-old infants were exposed to either a one-peaked or two-peaked distribution from a continuum of vowels that formed a contrast in English, but not in Dutch. We used eye tracking to measure effects of distribution and sensory modality on infants' discrimination of the contrast. Although there were no overall effects of distribution or modality, separate t-tests in each of the six training conditions demonstrated significant discrimination of the vowel contrast in the two-peaked multimodal condition. For the modalities where the mouth was visible (visual-only and multimodal) we further examined infant looking patterns for the dynamic speaker's face. Infants in the two-peaked multimodal condition looked longer at her mouth than infants in any of the three other conditions. We propose that by 8 months, infants' native vowel categories are established insofar that learning a novel contrast is supported by attention to additional information, such as visual articulations.

No MeSH data available.


Related in: MedlinePlus