Neuronal ensemble control of prosthetic devices by a human with tetraplegia
TL;DR: Initial results for a tetraplegic human using a pilot NMP suggest that NMPs based upon intracortical neuronal ensemble spiking activity could provide a valuable new neurotechnology to restore independence for humans with paralysis.
Abstract: Neuromotor prostheses (NMPs) aim to replace or restore lost motor functions in paralysed humans by routeing movement-related signals from the brain, around damaged parts of the nervous system, to external effectors. To translate preclinical results from intact animals to a clinically useful NMP, movement signals must persist in cortex after spinal cord injury and be engaged by movement intent when sensory inputs and limb movement are long absent. Furthermore, NMPs would require that intention-driven neuronal activity be converted into a control signal that enables useful tasks. Here we show initial results for a tetraplegic human (MN) using a pilot NMP. Neuronal ensemble activity recorded through a 96-microelectrode array implanted in primary motor cortex demonstrated that intended hand motion modulates cortical spiking patterns three years after spinal cord injury. Decoders were created, providing a ‘neural cursor’ with which MN opened simulated e-mail and operated devices such as a television, even while conversing. Furthermore, MN used neural control to open and close a prosthetic hand, and perform rudimentary actions with a multi-jointed robotic arm. These early results suggest that NMPs based upon intracortical neuronal ensemble spiking activity could provide a valuable new neurotechnology to restore independence for humans with paralysis. The cover shows Matt Nagle, first participant in the BrainGate pilot clinical trial. He is unable to move his arms or legs following cervical spinal cord injury. Researchers at the Department of Neuroscience at Brown University, working with biotech company Cyberkinetics and 3 other institutions, demonstrate that movement-related signals can be relayed from the brain via an implanted BrainGate chip, allowing the patient to drive a computer screen cursor and activate simple robotic devices. Such neuromotor prostheses could pave the way towards systems to replace or restore lost motor function in paralysed humans. Prior to this advance, this type of work has been performed chiefly in monkeys. The latest example of such research has achieved a large increase in speed over current devices, enhancing the prospects for clinically viable brain-machine interfaces.
Citations
More filters
TL;DR: The brain's electrical signals enable people without muscle control to physically interact with the world through the use of their brains' electrical signals.
Abstract: The brain's electrical signals enable people without muscle control to physically interact with the world.
2,361 citations
Cites background from "Neuronal ensemble control of prosth..."
...Much of the related research has been done in non-human primates, though trials have also been done with humans.(12) Other studies have shown that recordings of electrocorticographic (ECoG) activity from the surface of the brain can also provide signals for a BCI(15); to date they indicate that invasive recording methods can also serve as the basis for BCIs....
[...]
TL;DR: The results demonstrate the feasibility for people with tetraplegia, years after injury to the central nervous system, to recreate useful multidimensional control of complex devices directly from a small sample of neural signals.
Abstract: Two people with long-standing tetraplegia use neural interface system-based control of a robotic arm to perform three-dimensional reach and grasp movements. John Donoghue and colleagues have previously demonstrated that people with tetraplegia can learn to use neural signals from the motor cortex to control a computer cursor. Work from another lab has also shown that monkeys can learn to use such signals to feed themselves with a robotic arm. Now, Donoghue and colleagues have advanced the technology to a level at which two people with long-standing paralysis — a 58-year-old woman and a 66-year-old man — are able to use a neural interface to direct a robotic arm to reach for and grasp objects. One subject was able to learn to pick up and drink from a bottle using a device implanted 5 years earlier, demonstrating not only that subjects can use the brain–machine interface, but also that it has potential longevity. Paralysis following spinal cord injury, brainstem stroke, amyotrophic lateral sclerosis and other disorders can disconnect the brain from the body, eliminating the ability to perform volitional movements. A neural interface system1,2,3,4,5 could restore mobility and independence for people with paralysis by translating neuronal activity directly into control signals for assistive devices. We have previously shown that people with long-standing tetraplegia can use a neural interface system to move and click a computer cursor and to control physical devices6,7,8. Able-bodied monkeys have used a neural interface system to control a robotic arm9, but it is unknown whether people with profound upper extremity paralysis or limb loss could use cortical neuronal ensemble signals to direct useful arm actions. Here we demonstrate the ability of two people with long-standing tetraplegia to use neural interface system-based control of a robotic arm to perform three-dimensional reach and grasp movements. Participants controlled the arm and hand over a broad space without explicit training, using signals decoded from a small, local population of motor cortex (MI) neurons recorded from a 96-channel microelectrode array. One of the study participants, implanted with the sensor 5 years earlier, also used a robotic arm to drink coffee from a bottle. Although robotic reach and grasp actions were not as fast or accurate as those of an able-bodied person, our results demonstrate the feasibility for people with tetraplegia, years after injury to the central nervous system, to recreate useful multidimensional control of complex devices directly from a small sample of neural signals.
2,181 citations
TL;DR: A system that permits embodied prosthetic control is described and monkeys (Macaca mulatta) use their motor cortical activity to control a mechanized arm replica in a self-feeding task, and this demonstration of multi-degree-of-freedom embodied prosthetics control paves the way towards the development of dexterous prosthetic devices that could ultimately achieve arm and hand function at a near-natural level.
Abstract: Brain-machine interfaces have mostly been used previously to move cursors on computer displays. Now experiments on macaque monkeys show that brain activity signals can control a multi-jointed prosthetic device in real-time. The monkeys used motor cortical activity to control a human-like prosthetic arm in a self-feeding task, with a greater sophistication of control than previously possible. This work could be important for the development of more practical neuro-prosthetic devices in the future. A system where monkeys use their motor cortical activity to control a robotic arm in a real-time self-feeding task, showing a significantly greater sophisitication of control than in previous studies, is demonstrated. This work could be important for the development of more practical neuro-prosthetic devices in the future. Arm movement is well represented in populations of neurons recorded from the motor cortex1,2,3,4,5,6,7. Cortical activity patterns have been used in the new field of brain–machine interfaces8,9,10,11 to show how cursors on computer displays can be moved in two- and three-dimensional space12,13,14,15,16,17,18,19,20,21,22. Although the ability to move a cursor can be useful in its own right, this technology could be applied to restore arm and hand function for amputees and paralysed persons. However, the use of cortical signals to control a multi-jointed prosthetic device for direct real-time interaction with the physical environment (‘embodiment’) has not been demonstrated. Here we describe a system that permits embodied prosthetic control; we show how monkeys (Macaca mulatta) use their motor cortical activity to control a mechanized arm replica in a self-feeding task. In addition to the three dimensions of movement, the subjects’ cortical signals also proportionally controlled a gripper on the end of the arm. Owing to the physical interaction between the monkey, the robotic arm and objects in the workspace, this new task presented a higher level of difficulty than previous virtual (cursor-control) experiments. Apart from an example of simple one-dimensional control23, previous experiments have lacked physical interaction even in cases where a robotic arm16,19,24 or hand20 was included in the control loop, because the subjects did not use it to interact with physical objects—an interaction that cannot be fully simulated. This demonstration of multi-degree-of-freedom embodied prosthetic control paves the way towards the development of dexterous prosthetic devices that could ultimately achieve arm and hand function at a near-natural level.
1,579 citations
TL;DR: The state-of-the-art of BCIs are reviewed, looking at the different steps that form a standard BCI: signal acquisition, preprocessing or signal enhancement, feature extraction, classification and the control interface.
Abstract: A brain-computer interface (BCI) is a hardware and software communications system that permits cerebral activity alone to control computers or external devices. The immediate goal of BCI research is to provide communications capabilities to severely disabled people who are totally paralyzed or 'locked in' by neurological neuromuscular disorders, such as amyotrophic lateral sclerosis, brain stem stroke, or spinal cord injury. Here, we review the state-of-the-art of BCIs, looking at the different steps that form a standard BCI: signal acquisition, preprocessing or signal enhancement, feature extraction, classification and the control interface. We discuss their advantages, drawbacks, and latest advances, and we survey the numerous technologies reported in the scientific literature to design each step of a BCI. First, the review examines the neuroimaging modalities used in the signal acquisition step, each of which monitors a different functional brain activity such as electrical, magnetic or metabolic activity. Second, the review discusses different electrophysiological control signals that determine user intentions, which can be detected in brain activity. Third, the review includes some techniques used in the signal enhancement step to deal with the artifacts in the control signals and improve the performance. Fourth, the review studies some mathematic algorithms used in the feature extraction and classification steps which translate the information in the control signals into commands that operate a computer or other device. Finally, the review provides an overview of various BCI applications that control a range of devices.
1,407 citations
Additional excerpts
...[287]...
[...]
TL;DR: It is hypothesized that cell assemblies are best understood in light of their output product, as detected by "reader-actuator" mechanisms, and it is suggested that the hierarchical organization of cell assemblies may be regarded as a neural syntax.
1,105 citations
Cites background or methods from "Neuronal ensemble control of prosth..."
...NEURON 103 ters that best describe the executed movement (Carmena et al., 2003; Chapin et al., 1999; Hochberg et al., 2006; Taylor et al., 2002)....
[...]
...several dozens of simultaneously recorded neurons from either the motor cortex or other areas improve the prediction only by a modest 10% to 15% (Figure 5B; Carmena et al., 2003; Hochberg et al., 2006; Serruya et al., 2002; Taylor et al., 2002; Wessberg et al., 2000; cf., Nicolelis and Lebedev, 2009)....
[...]
...…dozens of simultaneously recorded neurons from either the motor cortex or other areas improve the prediction only by a modest 10% to 15% (Figure 5B; Carmena et al., 2003; Hochberg et al., 2006; Serruya et al., 2002; Taylor et al., 2002; Wessberg et al., 2000; cf., Nicolelis and Lebedev, 2009)....
[...]
...In this process, various statistical extraction methods are used to identify the conversion parame- 368 Neuron 68, November 4, 2010 ª2010 Elsevier Inc. ters that best describe the executed movement (Carmena et al., 2003; Chapin et al., 1999; Hochberg et al., 2006; Taylor et al., 2002)....
[...]
References
More filters
TL;DR: This is the first volume of the proposed many-sectioned "Handbook" in which the American Physiological Society intends to present comprehensively the entire field of physiology.
Abstract: This is the first volume of the proposed many-sectioned "Handbook" in which the American Physiological Society intends to present comprehensively the entire field of physiology. The scope and depth of the work may be estimated by realizing that the Section on Neurophysiology alone will comprise three volumes. The present work covers the following principal topics, all but the first of which are divided into several chapters: History of Neurophysiology (M. A. B. Brazier), Neuron Physiology (Introduction, J. C. Eccles), Brain Potentials and Rhythms (Introduction, A. Fessard), Sensory Mechanisms (Introduction, Lord Adrian), and Vision (Introduction, H. K. Hartline). To a large degree, the individual contributors have overcome the difficult task of presenting both the fundamentals and the most recent advances pertaining to their subjects. The historical development of knowledge and hypotheses about most of the particular topics is given detailed consideration, and this aspect alone should help to prevent the material
10,658 citations
01 Jun 2000
TL;DR: The first international meeting devoted to brain-computer interface research and development is summarized, which focuses on the development of appropriate applications, identification of appropriate user groups, and careful attention to the needs and desires of individual users.
Abstract: Over the past decade, many laboratories have begun to explore brain-computer interface (BCI) technology as a radically new communication option for those with neuromuscular impairments that prevent them from using conventional augmentative communication methods. BCI's provide these users with communication channels that do not depend on peripheral nerves and muscles. This article summarizes the first international meeting devoted to BCI research and development. Current BCI's use electroencephalographic (EEG) activity recorded at the scalp or single-unit activity recorded from within cortex to control cursor movement, select letters or icons, or operate a neuroprosthesis. The central element in each BCI is a translation algorithm that converts electrophysiological input from the user into output that controls external devices. BCI operation depends on effective interaction between two adaptive controllers, the user who encodes his or her commands in the electrophysiological input provided to the BCI, and the BCI which recognizes the commands contained in the input and expresses them in device control. Current BCI's have maximum information transfer rates of 5-25 b/min. Achievement of greater speed and accuracy depends on improvements in signal processing, translation algorithms, and user training. These improvements depend on increased interdisciplinary cooperation between neuroscientists, engineers, computer programmers, psychologists, and rehabilitation specialists, and on adoption and widespread application of objective methods for evaluating alternative methods. The practical use of BCI technology depends on the development of appropriate applications, identification of appropriate user groups, and careful attention to the needs and desires of individual users. BCI research and development will also benefit from greater emphasis on peer-reviewed publications, and from adoption of standard venues for presentations and discussion.
2,121 citations
TL;DR: International Standards for Neurological and Functional Classification of Spinal Cord Injury are published and will be used for clinical practice.
Abstract: International Standards for Neurological and Functional Classification of Spinal Cord Injury
2,062 citations
TL;DR: The orderly variation of cell discharge with the direction of movement and the fact that cells related to only one of the eight directions of movement tested were rarely observed indicate that movements in a particular direction are not subserved by motor cortical cells uniquely related to that movement.
Abstract: The activity of single cells in the motor cortex was recorded while monkeys made arm movements in eight directions (at 45 degrees intervals) in a two-dimensional apparatus. These movements started from the same point and were of the same amplitude. The activity of 606 cells related to proximal arm movements was examined in the task; 323 of the 606 cells were active in that task and were studied in detail. The frequency of discharge of 241 of the 323 cells (74.6%) varied in an orderly fashion with the direction of movement. Discharge was most intense with movements in a preferred direction and was reduced gradually when movements were made in directions farther and farther away from the preferred one. This resulted in a bell-shaped directional tuning curve. These relations were observed for cell discharge during the reaction time, the movement time, and the period that preceded the earliest changes in the electromyographic activity (approximately 80 msec before movement onset). In about 75% of the 241 directionally tuned cells, the frequency of discharge, D, was a sinusoidal function of the direction of movement, theta: D = b0 + b1 sin theta + b2cos theta, or, in terms of the preferred direction, theta 0: D = b0 + c1cos (theta - theta0), where b0, b1, b2, and c1 are regression coefficients. Preferred directions differed for different cells so that the tuning curves partially overlapped. The orderly variation of cell discharge with the direction of movement and the fact that cells related to only one of the eight directions of movement tested were rarely observed indicate that movements in a particular direction are not subserved by motor cortical cells uniquely related to that movement. It is suggested, instead, that a movement trajectory in a desired direction might be generated by the cooperation of cells with overlapping tuning curves. The nature of this hypothetical population code for movement direction remains to be elucidated.
2,049 citations
TL;DR: It is demonstrated that primates can learn to reach and grasp virtual objects by controlling a robot arm through a closed-loop brain–machine interface (BMIc) that uses multiple mathematical models to extract several motor parameters from the electrical activity of frontoparietal neuronal ensembles.
Abstract: Reaching and grasping in primates depend on the coordination of neural activity in large frontoparietal ensembles. Here we demonstrate that primates can learn to reach and grasp virtual objects by controlling a robot arm through a closed-loop brain–machine interface (BMIc) that uses multiple mathematical models to extract several motor parameters (i.e., hand position, velocity, gripping force, and the EMGs of multiple arm muscles) from the electrical activity of frontoparietal neuronal ensembles. As single neurons typically contribute to the encoding of several motor parameters, we observed that high BMIc accuracy required recording from large neuronal ensembles. Continuous BMIc operation by monkeys led to significant improvements in both model predictions and behavioral performance. Using visual feedback, monkeys succeeded in producing robot reach-and-grasp movements even when their arms did not move. Learning to operate the BMIc was paralleled by functional reorganization in multiple cortical areas, suggesting that the dynamic properties of the BMIc were incorporated into motor and sensory cortical representations.
1,740 citations