Limits...
Learning to control a brain-machine interface for reaching and grasping by primates.

Carmena JM, Lebedev MA, Crist RE, O'Doherty JE, Santucci DM, Dimitrov DF, Patil PG, Henriquez CS, Nicolelis MA - PLoS Biol. (2003)

Bottom Line: Continuous BMIc operation by monkeys led to significant improvements in both model predictions and behavioral performance.Using visual feedback, monkeys succeeded in producing robot reach-and-grasp movements even when their arms did not move.Learning to operate the BMIc was paralleled by functional reorganization in multiple cortical areas, suggesting that the dynamic properties of the BMIc were incorporated into motor and sensory cortical representations.

View Article: PubMed Central - PubMed

Affiliation: Department of Neurobiology, Duke University, Durham, North Carolina, USA.

ABSTRACT
Reaching and grasping in primates depend on the coordination of neural activity in large frontoparietal ensembles. Here we demonstrate that primates can learn to reach and grasp virtual objects by controlling a robot arm through a closed-loop brain-machine interface (BMIc) that uses multiple mathematical models to extract several motor parameters (i.e., hand position, velocity, gripping force, and the EMGs of multiple arm muscles) from the electrical activity of frontoparietal neuronal ensembles. As single neurons typically contribute to the encoding of several motor parameters, we observed that high BMIc accuracy required recording from large neuronal ensembles. Continuous BMIc operation by monkeys led to significant improvements in both model predictions and behavioral performance. Using visual feedback, monkeys succeeded in producing robot reach-and-grasp movements even when their arms did not move. Learning to operate the BMIc was paralleled by functional reorganization in multiple cortical areas, suggesting that the dynamic properties of the BMIc were incorporated into motor and sensory cortical representations.

Show MeSH
Long-Term Functional Changes in Multiple Cortical Areas(A) Color-coded (red shows high values; blue, low values) representation of individual contributions measured as the correlation coefficient (R) of neurons to linear model predictions of hand position for 42 training sessions. The average contribution steadily increased with training. The bar on the left indicates cortical location of the neurons.(B–E) Average contribution of neurons located in different cortical areas (PMd, M1, S1, and SMA, respectively) to hand position prediction during 42 recording sessions.(F) Average contribution for the whole ensemble to hand position prediction versus hand velocity predictions. A linear increase in contribution was observed only for predictions of hand position.
© Copyright Policy
Related In: Results  -  Collection


getmorefigures.php?uid=PMC261882&req=5

pbio.0000042-g003: Long-Term Functional Changes in Multiple Cortical Areas(A) Color-coded (red shows high values; blue, low values) representation of individual contributions measured as the correlation coefficient (R) of neurons to linear model predictions of hand position for 42 training sessions. The average contribution steadily increased with training. The bar on the left indicates cortical location of the neurons.(B–E) Average contribution of neurons located in different cortical areas (PMd, M1, S1, and SMA, respectively) to hand position prediction during 42 recording sessions.(F) Average contribution for the whole ensemble to hand position prediction versus hand velocity predictions. A linear increase in contribution was observed only for predictions of hand position.

Mentions: Long-term functional changes in multiple cortical areas were evident in both animals. For instance, the average contribution of single neurons to model performance increased with learning. Figure 3A shows changes in the contribution of single cortical neurons (measured in terms of correlation coefficient, R, color-coded, where blue shows low R; red, high R) from five cortical areas (PMd, M1, S1, SMA, and M1 ipsilateral) to the linear model that predicted hand position in task 1. Data from 42 recording sessions are shown. In these sessions, predictions of hand position (HPx, HPy) were used to control the cursor on the screen. By the end of the training, very accurate predictions of hand position and velocity were obtained (mean R ± SEM; HPx = 0.75 ± 0.04, HPy = 0.72 ± 0.04, HVx = 0.70 ± 0.03, and HVy = 0.71 ± 0.02). These high values were reached through a significant increase in contribution of individual neurons to the linear model. When the mean contribution of single neurons was plotted as a function of their cortical area location, differences across cortical areas were found (Figure 3B–3E). The change was higher in SMA (Figure 3E; R = 0.81, slope = 0.01, p < 0.001) than in PMd (Figure 3B; R = 0.81, slope = 1 × 10−3, p < 0.001), S1 (Figure 3D; R = 0.67, slope = 4 × 10−3, p < 0.001), and M1 (Figure 3C; R = 0.50, slope = 3 × 10−3, p < 0.001). Note that from the beginning of training, M1 neurons (Figure 3C) provided the highest mean contribution. By the end of 42 sessions, however, the mean contribution of neurons located in other cortical areas (e.g., SMA, PMd, and S1) was as high as that of M1. It is noteworthy that the significant enhancement in contribution occurred for the model predicting hand position (average of all cortical areas, R = 0.80, slope = 4 × 10−3, p < 0.001), but not the one predicting hand velocity (R = 0.05, slope = 2.2 × 10−4). This selectivity coincided with the use of a position model in the BMIc during these 42 sessions. Thus, long-term training with the BMIc using a particular model could result in selective enhancement of the mean contribution of neurons to that model, but not the others.


Learning to control a brain-machine interface for reaching and grasping by primates.

Carmena JM, Lebedev MA, Crist RE, O'Doherty JE, Santucci DM, Dimitrov DF, Patil PG, Henriquez CS, Nicolelis MA - PLoS Biol. (2003)

Long-Term Functional Changes in Multiple Cortical Areas(A) Color-coded (red shows high values; blue, low values) representation of individual contributions measured as the correlation coefficient (R) of neurons to linear model predictions of hand position for 42 training sessions. The average contribution steadily increased with training. The bar on the left indicates cortical location of the neurons.(B–E) Average contribution of neurons located in different cortical areas (PMd, M1, S1, and SMA, respectively) to hand position prediction during 42 recording sessions.(F) Average contribution for the whole ensemble to hand position prediction versus hand velocity predictions. A linear increase in contribution was observed only for predictions of hand position.
© Copyright Policy
Related In: Results  -  Collection

Show All Figures
getmorefigures.php?uid=PMC261882&req=5

pbio.0000042-g003: Long-Term Functional Changes in Multiple Cortical Areas(A) Color-coded (red shows high values; blue, low values) representation of individual contributions measured as the correlation coefficient (R) of neurons to linear model predictions of hand position for 42 training sessions. The average contribution steadily increased with training. The bar on the left indicates cortical location of the neurons.(B–E) Average contribution of neurons located in different cortical areas (PMd, M1, S1, and SMA, respectively) to hand position prediction during 42 recording sessions.(F) Average contribution for the whole ensemble to hand position prediction versus hand velocity predictions. A linear increase in contribution was observed only for predictions of hand position.
Mentions: Long-term functional changes in multiple cortical areas were evident in both animals. For instance, the average contribution of single neurons to model performance increased with learning. Figure 3A shows changes in the contribution of single cortical neurons (measured in terms of correlation coefficient, R, color-coded, where blue shows low R; red, high R) from five cortical areas (PMd, M1, S1, SMA, and M1 ipsilateral) to the linear model that predicted hand position in task 1. Data from 42 recording sessions are shown. In these sessions, predictions of hand position (HPx, HPy) were used to control the cursor on the screen. By the end of the training, very accurate predictions of hand position and velocity were obtained (mean R ± SEM; HPx = 0.75 ± 0.04, HPy = 0.72 ± 0.04, HVx = 0.70 ± 0.03, and HVy = 0.71 ± 0.02). These high values were reached through a significant increase in contribution of individual neurons to the linear model. When the mean contribution of single neurons was plotted as a function of their cortical area location, differences across cortical areas were found (Figure 3B–3E). The change was higher in SMA (Figure 3E; R = 0.81, slope = 0.01, p < 0.001) than in PMd (Figure 3B; R = 0.81, slope = 1 × 10−3, p < 0.001), S1 (Figure 3D; R = 0.67, slope = 4 × 10−3, p < 0.001), and M1 (Figure 3C; R = 0.50, slope = 3 × 10−3, p < 0.001). Note that from the beginning of training, M1 neurons (Figure 3C) provided the highest mean contribution. By the end of 42 sessions, however, the mean contribution of neurons located in other cortical areas (e.g., SMA, PMd, and S1) was as high as that of M1. It is noteworthy that the significant enhancement in contribution occurred for the model predicting hand position (average of all cortical areas, R = 0.80, slope = 4 × 10−3, p < 0.001), but not the one predicting hand velocity (R = 0.05, slope = 2.2 × 10−4). This selectivity coincided with the use of a position model in the BMIc during these 42 sessions. Thus, long-term training with the BMIc using a particular model could result in selective enhancement of the mean contribution of neurons to that model, but not the others.

Bottom Line: Continuous BMIc operation by monkeys led to significant improvements in both model predictions and behavioral performance.Using visual feedback, monkeys succeeded in producing robot reach-and-grasp movements even when their arms did not move.Learning to operate the BMIc was paralleled by functional reorganization in multiple cortical areas, suggesting that the dynamic properties of the BMIc were incorporated into motor and sensory cortical representations.

View Article: PubMed Central - PubMed

Affiliation: Department of Neurobiology, Duke University, Durham, North Carolina, USA.

ABSTRACT
Reaching and grasping in primates depend on the coordination of neural activity in large frontoparietal ensembles. Here we demonstrate that primates can learn to reach and grasp virtual objects by controlling a robot arm through a closed-loop brain-machine interface (BMIc) that uses multiple mathematical models to extract several motor parameters (i.e., hand position, velocity, gripping force, and the EMGs of multiple arm muscles) from the electrical activity of frontoparietal neuronal ensembles. As single neurons typically contribute to the encoding of several motor parameters, we observed that high BMIc accuracy required recording from large neuronal ensembles. Continuous BMIc operation by monkeys led to significant improvements in both model predictions and behavioral performance. Using visual feedback, monkeys succeeded in producing robot reach-and-grasp movements even when their arms did not move. Learning to operate the BMIc was paralleled by functional reorganization in multiple cortical areas, suggesting that the dynamic properties of the BMIc were incorporated into motor and sensory cortical representations.

Show MeSH