Limits...
Learning to control a brain-machine interface for reaching and grasping by primates.

Carmena JM, Lebedev MA, Crist RE, O'Doherty JE, Santucci DM, Dimitrov DF, Patil PG, Henriquez CS, Nicolelis MA - PLoS Biol. (2003)

Bottom Line: Continuous BMIc operation by monkeys led to significant improvements in both model predictions and behavioral performance.Using visual feedback, monkeys succeeded in producing robot reach-and-grasp movements even when their arms did not move.Learning to operate the BMIc was paralleled by functional reorganization in multiple cortical areas, suggesting that the dynamic properties of the BMIc were incorporated into motor and sensory cortical representations.

View Article: PubMed Central - PubMed

Affiliation: Department of Neurobiology, Duke University, Durham, North Carolina, USA.

ABSTRACT
Reaching and grasping in primates depend on the coordination of neural activity in large frontoparietal ensembles. Here we demonstrate that primates can learn to reach and grasp virtual objects by controlling a robot arm through a closed-loop brain-machine interface (BMIc) that uses multiple mathematical models to extract several motor parameters (i.e., hand position, velocity, gripping force, and the EMGs of multiple arm muscles) from the electrical activity of frontoparietal neuronal ensembles. As single neurons typically contribute to the encoding of several motor parameters, we observed that high BMIc accuracy required recording from large neuronal ensembles. Continuous BMIc operation by monkeys led to significant improvements in both model predictions and behavioral performance. Using visual feedback, monkeys succeeded in producing robot reach-and-grasp movements even when their arms did not move. Learning to operate the BMIc was paralleled by functional reorganization in multiple cortical areas, suggesting that the dynamic properties of the BMIc were incorporated into motor and sensory cortical representations.

Show MeSH
Experimental Setup, Behavioral Tasks, Changes in Performance with Training, EMG Records during Pole and Brain Control, and Stability of Model Predictions(A) Behavioral setup and control loops, consisting of the data acquisition system, the computer running multiple linear models in real time, the robot arm equipped with a gripper, and the visual display. The pole was equipped with a gripping force transducer. Robot position was translated into cursor position on the screen, and feedback of the gripping force was provided by changing the cursor size.(B) Schematics of three behavioral tasks. In task 1, the monkey's goal was to move the cursor to a visual target (green) that appeared at random locations on the screen. In task 2, the pole was stationary, and the monkey had to grasp a virtual object by developing a particular gripping force instructed by two red circles displayed on the screen. Task 3 was a combination of tasks 1 and 2. The monkey had to move the cursor to the target and then develop a gripping force necessary to grasp a virtual object.(C–E) Behavioral performance for two monkeys in tasks 1–3. The percentage of correctly completed trials increased, while the time to conclude a trial decreased with training. This was true for both pole (blue) and brain (red) control. Horizontal (green) lines indicate chance performance obtained from the random walk model. The introduction of the robot arm into the BMIc control loop resulted in a drop in behavioral performance. In approximately seven training sessions, the animal's behavioral performance gradually returned to the initial values. This effect took place during both pole and brain control.(F) Stability of model predictions of hand velocity during long pole-control sessions (more than 50 min) for two monkeys performing task 1. The first 10 min of performance were used to train the model, and then its coefficients were frozen. Model predictions remained highly accurate for tens of minutes.(G) Surface EMGs of arm muscles recorded in task 1 for pole control (left) and brain control without arm movements (right). Top plots show the X-coordinate of the cursor; plots below display EMGs of wrist flexors, wrist extensors, and biceps. EMG modulations were absent in brain control.
© Copyright Policy
Related In: Results  -  Collection


getmorefigures.php?uid=PMC261882&req=5

pbio.0000042-g001: Experimental Setup, Behavioral Tasks, Changes in Performance with Training, EMG Records during Pole and Brain Control, and Stability of Model Predictions(A) Behavioral setup and control loops, consisting of the data acquisition system, the computer running multiple linear models in real time, the robot arm equipped with a gripper, and the visual display. The pole was equipped with a gripping force transducer. Robot position was translated into cursor position on the screen, and feedback of the gripping force was provided by changing the cursor size.(B) Schematics of three behavioral tasks. In task 1, the monkey's goal was to move the cursor to a visual target (green) that appeared at random locations on the screen. In task 2, the pole was stationary, and the monkey had to grasp a virtual object by developing a particular gripping force instructed by two red circles displayed on the screen. Task 3 was a combination of tasks 1 and 2. The monkey had to move the cursor to the target and then develop a gripping force necessary to grasp a virtual object.(C–E) Behavioral performance for two monkeys in tasks 1–3. The percentage of correctly completed trials increased, while the time to conclude a trial decreased with training. This was true for both pole (blue) and brain (red) control. Horizontal (green) lines indicate chance performance obtained from the random walk model. The introduction of the robot arm into the BMIc control loop resulted in a drop in behavioral performance. In approximately seven training sessions, the animal's behavioral performance gradually returned to the initial values. This effect took place during both pole and brain control.(F) Stability of model predictions of hand velocity during long pole-control sessions (more than 50 min) for two monkeys performing task 1. The first 10 min of performance were used to train the model, and then its coefficients were frozen. Model predictions remained highly accurate for tens of minutes.(G) Surface EMGs of arm muscles recorded in task 1 for pole control (left) and brain control without arm movements (right). Top plots show the X-coordinate of the cursor; plots below display EMGs of wrist flexors, wrist extensors, and biceps. EMG modulations were absent in brain control.

Mentions: Using the experimental apparatus illustrated in Figure 1A, monkeys were trained in three different tasks: a reaching task (task 1; Figure 1B), a hand-gripping task (task 2; Figure 1B), and a reach-and-grasp task (task 3; Figure 1B).


Learning to control a brain-machine interface for reaching and grasping by primates.

Carmena JM, Lebedev MA, Crist RE, O'Doherty JE, Santucci DM, Dimitrov DF, Patil PG, Henriquez CS, Nicolelis MA - PLoS Biol. (2003)

Experimental Setup, Behavioral Tasks, Changes in Performance with Training, EMG Records during Pole and Brain Control, and Stability of Model Predictions(A) Behavioral setup and control loops, consisting of the data acquisition system, the computer running multiple linear models in real time, the robot arm equipped with a gripper, and the visual display. The pole was equipped with a gripping force transducer. Robot position was translated into cursor position on the screen, and feedback of the gripping force was provided by changing the cursor size.(B) Schematics of three behavioral tasks. In task 1, the monkey's goal was to move the cursor to a visual target (green) that appeared at random locations on the screen. In task 2, the pole was stationary, and the monkey had to grasp a virtual object by developing a particular gripping force instructed by two red circles displayed on the screen. Task 3 was a combination of tasks 1 and 2. The monkey had to move the cursor to the target and then develop a gripping force necessary to grasp a virtual object.(C–E) Behavioral performance for two monkeys in tasks 1–3. The percentage of correctly completed trials increased, while the time to conclude a trial decreased with training. This was true for both pole (blue) and brain (red) control. Horizontal (green) lines indicate chance performance obtained from the random walk model. The introduction of the robot arm into the BMIc control loop resulted in a drop in behavioral performance. In approximately seven training sessions, the animal's behavioral performance gradually returned to the initial values. This effect took place during both pole and brain control.(F) Stability of model predictions of hand velocity during long pole-control sessions (more than 50 min) for two monkeys performing task 1. The first 10 min of performance were used to train the model, and then its coefficients were frozen. Model predictions remained highly accurate for tens of minutes.(G) Surface EMGs of arm muscles recorded in task 1 for pole control (left) and brain control without arm movements (right). Top plots show the X-coordinate of the cursor; plots below display EMGs of wrist flexors, wrist extensors, and biceps. EMG modulations were absent in brain control.
© Copyright Policy
Related In: Results  -  Collection

Show All Figures
getmorefigures.php?uid=PMC261882&req=5

pbio.0000042-g001: Experimental Setup, Behavioral Tasks, Changes in Performance with Training, EMG Records during Pole and Brain Control, and Stability of Model Predictions(A) Behavioral setup and control loops, consisting of the data acquisition system, the computer running multiple linear models in real time, the robot arm equipped with a gripper, and the visual display. The pole was equipped with a gripping force transducer. Robot position was translated into cursor position on the screen, and feedback of the gripping force was provided by changing the cursor size.(B) Schematics of three behavioral tasks. In task 1, the monkey's goal was to move the cursor to a visual target (green) that appeared at random locations on the screen. In task 2, the pole was stationary, and the monkey had to grasp a virtual object by developing a particular gripping force instructed by two red circles displayed on the screen. Task 3 was a combination of tasks 1 and 2. The monkey had to move the cursor to the target and then develop a gripping force necessary to grasp a virtual object.(C–E) Behavioral performance for two monkeys in tasks 1–3. The percentage of correctly completed trials increased, while the time to conclude a trial decreased with training. This was true for both pole (blue) and brain (red) control. Horizontal (green) lines indicate chance performance obtained from the random walk model. The introduction of the robot arm into the BMIc control loop resulted in a drop in behavioral performance. In approximately seven training sessions, the animal's behavioral performance gradually returned to the initial values. This effect took place during both pole and brain control.(F) Stability of model predictions of hand velocity during long pole-control sessions (more than 50 min) for two monkeys performing task 1. The first 10 min of performance were used to train the model, and then its coefficients were frozen. Model predictions remained highly accurate for tens of minutes.(G) Surface EMGs of arm muscles recorded in task 1 for pole control (left) and brain control without arm movements (right). Top plots show the X-coordinate of the cursor; plots below display EMGs of wrist flexors, wrist extensors, and biceps. EMG modulations were absent in brain control.
Mentions: Using the experimental apparatus illustrated in Figure 1A, monkeys were trained in three different tasks: a reaching task (task 1; Figure 1B), a hand-gripping task (task 2; Figure 1B), and a reach-and-grasp task (task 3; Figure 1B).

Bottom Line: Continuous BMIc operation by monkeys led to significant improvements in both model predictions and behavioral performance.Using visual feedback, monkeys succeeded in producing robot reach-and-grasp movements even when their arms did not move.Learning to operate the BMIc was paralleled by functional reorganization in multiple cortical areas, suggesting that the dynamic properties of the BMIc were incorporated into motor and sensory cortical representations.

View Article: PubMed Central - PubMed

Affiliation: Department of Neurobiology, Duke University, Durham, North Carolina, USA.

ABSTRACT
Reaching and grasping in primates depend on the coordination of neural activity in large frontoparietal ensembles. Here we demonstrate that primates can learn to reach and grasp virtual objects by controlling a robot arm through a closed-loop brain-machine interface (BMIc) that uses multiple mathematical models to extract several motor parameters (i.e., hand position, velocity, gripping force, and the EMGs of multiple arm muscles) from the electrical activity of frontoparietal neuronal ensembles. As single neurons typically contribute to the encoding of several motor parameters, we observed that high BMIc accuracy required recording from large neuronal ensembles. Continuous BMIc operation by monkeys led to significant improvements in both model predictions and behavioral performance. Using visual feedback, monkeys succeeded in producing robot reach-and-grasp movements even when their arms did not move. Learning to operate the BMIc was paralleled by functional reorganization in multiple cortical areas, suggesting that the dynamic properties of the BMIc were incorporated into motor and sensory cortical representations.

Show MeSH