Limits...
Multisensory training can promote or impede visual perceptual learning of speech stimuli: visual-tactile vs. visual-auditory training.

Eberhardt SP, Auer ET, Bernstein LE - Front Hum Neurosci (2014)

Bottom Line: Here, impeder and promoter effects were sought in normal-hearing adults who participated in lipreading training.Across this and previous studies, multisensory training effects depended on the functional relationship between pathways engaged during training.Two principles are proposed to account for stimulus effects: (1) Stimuli presented to the trainee's primary perceptual pathway will impede learning by a lower-rank pathway. (2) Stimuli presented to the trainee's lower rank perceptual pathway will promote learning by a higher-rank pathway.

View Article: PubMed Central - PubMed

Affiliation: Communication Neuroscience Laboratory, Department of Speech and Hearing Sciences, George Washington University Washington, DC, USA.

ABSTRACT
In a series of studies we have been investigating how multisensory training affects unisensory perceptual learning with speech stimuli. Previously, we reported that audiovisual (AV) training with speech stimuli can promote auditory-only (AO) perceptual learning in normal-hearing adults but can impede learning in congenitally deaf adults with late-acquired cochlear implants. Here, impeder and promoter effects were sought in normal-hearing adults who participated in lipreading training. In Experiment 1, visual-only (VO) training on paired associations between CVCVC nonsense word videos and nonsense pictures demonstrated that VO words could be learned to a high level of accuracy even by poor lipreaders. In Experiment 2, visual-auditory (VA) training in the same paradigm but with the addition of synchronous vocoded acoustic speech impeded VO learning of the stimuli in the paired-associates paradigm. In Experiment 3, the vocoded AO stimuli were shown to be less informative than the VO speech. Experiment 4 combined vibrotactile speech stimuli with the visual stimuli during training. Vibrotactile stimuli were shown to promote visual perceptual learning. In Experiment 5, no-training controls were used to show that training with visual speech carried over to consonant identification of untrained CVCVC stimuli but not to lipreading words in sentences. Across this and previous studies, multisensory training effects depended on the functional relationship between pathways engaged during training. Two principles are proposed to account for stimulus effects: (1) Stimuli presented to the trainee's primary perceptual pathway will impede learning by a lower-rank pathway. (2) Stimuli presented to the trainee's lower rank perceptual pathway will promote learning by a higher-rank pathway. The mechanisms supporting these principles are discussed in light of multisensory reverse hierarchy theory (RHT).

No MeSH data available.


Sequence of events during a paired-associates training trial. The speech stimulus was presented, followed by the matrix of fribble responses, followed by the participant’s response selection, followed by feedback and a repetition of the speech stimulus. The next trial was initiated by clicking on the correct highlighted fribble. Panels depict the stimuli for the four experiments from left to right. The trial structure for VO (Exp. 1), VA (Exp. 2), AO (Exp. 3), and VT (Exp. 4) training followed the same sequence, except that during AO training with vocoded audio the participant was not shown a moving face. During VA the visual stimuli were accompanied by vocoded audio, and during VT the visual stimuli were accompanied by vibrotactile vocoder stimuli (adapted from Bernstein et al., 2013).
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4215828&req=5

Figure 2: Sequence of events during a paired-associates training trial. The speech stimulus was presented, followed by the matrix of fribble responses, followed by the participant’s response selection, followed by feedback and a repetition of the speech stimulus. The next trial was initiated by clicking on the correct highlighted fribble. Panels depict the stimuli for the four experiments from left to right. The trial structure for VO (Exp. 1), VA (Exp. 2), AO (Exp. 3), and VT (Exp. 4) training followed the same sequence, except that during AO training with vocoded audio the participant was not shown a moving face. During VA the visual stimuli were accompanied by vocoded audio, and during VT the visual stimuli were accompanied by vibrotactile vocoder stimuli (adapted from Bernstein et al., 2013).

Mentions: Figure 2 outlines the events within a paired-associates training trial. During training, the participant’s task was to learn, by trial and error with feedback on each trial, lists of individual associations between each of 12 CVCVC spoken nonsense words and 12 fribble images. The figure shows the four different types of training conditions, VO (Exp. 1), VA (Exp. 2), AO (Exp. 3), and VT (Exp. 4). Figure 2 shows that each trial was initiated by presenting a speech stimulus then the 12-fribble image matrix (3 rows of 4 columns, with image position within the matrix randomly selected on a trial-by-trial basis). The participant selected a fribble image and the screen darkened except for the correct response. The participant received the stimulus again, and after the stimulus was presented again, the participant clicked on the correct fribble in order to move on to the next trial.


Multisensory training can promote or impede visual perceptual learning of speech stimuli: visual-tactile vs. visual-auditory training.

Eberhardt SP, Auer ET, Bernstein LE - Front Hum Neurosci (2014)

Sequence of events during a paired-associates training trial. The speech stimulus was presented, followed by the matrix of fribble responses, followed by the participant’s response selection, followed by feedback and a repetition of the speech stimulus. The next trial was initiated by clicking on the correct highlighted fribble. Panels depict the stimuli for the four experiments from left to right. The trial structure for VO (Exp. 1), VA (Exp. 2), AO (Exp. 3), and VT (Exp. 4) training followed the same sequence, except that during AO training with vocoded audio the participant was not shown a moving face. During VA the visual stimuli were accompanied by vocoded audio, and during VT the visual stimuli were accompanied by vibrotactile vocoder stimuli (adapted from Bernstein et al., 2013).
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4215828&req=5

Figure 2: Sequence of events during a paired-associates training trial. The speech stimulus was presented, followed by the matrix of fribble responses, followed by the participant’s response selection, followed by feedback and a repetition of the speech stimulus. The next trial was initiated by clicking on the correct highlighted fribble. Panels depict the stimuli for the four experiments from left to right. The trial structure for VO (Exp. 1), VA (Exp. 2), AO (Exp. 3), and VT (Exp. 4) training followed the same sequence, except that during AO training with vocoded audio the participant was not shown a moving face. During VA the visual stimuli were accompanied by vocoded audio, and during VT the visual stimuli were accompanied by vibrotactile vocoder stimuli (adapted from Bernstein et al., 2013).
Mentions: Figure 2 outlines the events within a paired-associates training trial. During training, the participant’s task was to learn, by trial and error with feedback on each trial, lists of individual associations between each of 12 CVCVC spoken nonsense words and 12 fribble images. The figure shows the four different types of training conditions, VO (Exp. 1), VA (Exp. 2), AO (Exp. 3), and VT (Exp. 4). Figure 2 shows that each trial was initiated by presenting a speech stimulus then the 12-fribble image matrix (3 rows of 4 columns, with image position within the matrix randomly selected on a trial-by-trial basis). The participant selected a fribble image and the screen darkened except for the correct response. The participant received the stimulus again, and after the stimulus was presented again, the participant clicked on the correct fribble in order to move on to the next trial.

Bottom Line: Here, impeder and promoter effects were sought in normal-hearing adults who participated in lipreading training.Across this and previous studies, multisensory training effects depended on the functional relationship between pathways engaged during training.Two principles are proposed to account for stimulus effects: (1) Stimuli presented to the trainee's primary perceptual pathway will impede learning by a lower-rank pathway. (2) Stimuli presented to the trainee's lower rank perceptual pathway will promote learning by a higher-rank pathway.

View Article: PubMed Central - PubMed

Affiliation: Communication Neuroscience Laboratory, Department of Speech and Hearing Sciences, George Washington University Washington, DC, USA.

ABSTRACT
In a series of studies we have been investigating how multisensory training affects unisensory perceptual learning with speech stimuli. Previously, we reported that audiovisual (AV) training with speech stimuli can promote auditory-only (AO) perceptual learning in normal-hearing adults but can impede learning in congenitally deaf adults with late-acquired cochlear implants. Here, impeder and promoter effects were sought in normal-hearing adults who participated in lipreading training. In Experiment 1, visual-only (VO) training on paired associations between CVCVC nonsense word videos and nonsense pictures demonstrated that VO words could be learned to a high level of accuracy even by poor lipreaders. In Experiment 2, visual-auditory (VA) training in the same paradigm but with the addition of synchronous vocoded acoustic speech impeded VO learning of the stimuli in the paired-associates paradigm. In Experiment 3, the vocoded AO stimuli were shown to be less informative than the VO speech. Experiment 4 combined vibrotactile speech stimuli with the visual stimuli during training. Vibrotactile stimuli were shown to promote visual perceptual learning. In Experiment 5, no-training controls were used to show that training with visual speech carried over to consonant identification of untrained CVCVC stimuli but not to lipreading words in sentences. Across this and previous studies, multisensory training effects depended on the functional relationship between pathways engaged during training. Two principles are proposed to account for stimulus effects: (1) Stimuli presented to the trainee's primary perceptual pathway will impede learning by a lower-rank pathway. (2) Stimuli presented to the trainee's lower rank perceptual pathway will promote learning by a higher-rank pathway. The mechanisms supporting these principles are discussed in light of multisensory reverse hierarchy theory (RHT).

No MeSH data available.