Two people with long-standing tetraplegia use neural interface system-based control of a robotic arm to perform three-dimensional reach and grasp movements. John Donoghue and colleagues have previously demonstrated that people with tetraplegia can learn to use neural signals from the motor cortex to control a computer cursor. Work from another lab has also shown that monkeys can learn to use such signals to feed themselves with a robotic arm. Now, Donoghue and colleagues have advanced the technology to a level at which two people with long-standing paralysis — a 58-year-old woman and a 66-year-old man — are able to use a neural interface to direct a robotic arm to reach for and grasp objects. One subject was able to learn to pick up and drink from a bottle using a device implanted 5 years earlier, demonstrating not only that subjects can use the brain–machine interface, but also that it has potential longevity. Paralysis following spinal cord injury, brainstem stroke, amyotrophic lateral sclerosis and other disorders can disconnect the brain from the body, eliminating the ability to perform volitional movements. A neural interface system1,2,3,4,5 could restore mobility and independence for people with paralysis by translating neuronal activity directly into control signals for assistive devices. We have previously shown that people with long-standing tetraplegia can use a neural interface system to move and click a computer cursor and to control physical devices6,7,8. Able-bodied monkeys have used a neural interface system to control a robotic arm9, but it is unknown whether people with profound upper extremity paralysis or limb loss could use cortical neuronal ensemble signals to direct useful arm actions. Here we demonstrate the ability of two people with long-standing tetraplegia to use neural interface system-based control of a robotic arm to perform three-dimensional reach and grasp movements. Participants controlled the arm and hand over a broad space without explicit training, using signals decoded from a small, local population of motor cortex (MI) neurons recorded from a 96-channel microelectrode array. One of the study participants, implanted with the sensor 5 years earlier, also used a robotic arm to drink coffee from a bottle. Although robotic reach and grasp actions were not as fast or accurate as those of an able-bodied person, our results demonstrate the feasibility for people with tetraplegia, years after injury to the central nervous system, to recreate useful multidimensional control of complex devices directly from a small sample of neural signals.

/pdf/reach-and-grasp-by-people-with-tetraplegia-using-a-neurally-u1xyn3on0d.pdf

Reach and grasp by people with tetraplegia using a neurally controlled robotic arm

Brain-machine interfaces have mostly been used previously to move cursors on computer displays. Now experiments on macaque monkeys show that brain activity signals can control a multi-jointed prosthetic device in real-time. The monkeys used motor cortical activity to control a human-like prosthetic arm in a self-feeding task, with a greater sophistication of control than previously possible. This work could be important for the development of more practical neuro-prosthetic devices in the future. A system where monkeys use their motor cortical activity to control a robotic arm in a real-time self-feeding task, showing a significantly greater sophisitication of control than in previous studies, is demonstrated. This work could be important for the development of more practical neuro-prosthetic devices in the future. Arm movement is well represented in populations of neurons recorded from the motor cortex1,2,3,4,5,6,7. Cortical activity patterns have been used in the new field of brain–machine interfaces8,9,10,11 to show how cursors on computer displays can be moved in two- and three-dimensional space12,13,14,15,16,17,18,19,20,21,22. Although the ability to move a cursor can be useful in its own right, this technology could be applied to restore arm and hand function for amputees and paralysed persons. However, the use of cortical signals to control a multi-jointed prosthetic device for direct real-time interaction with the physical environment (‘embodiment’) has not been demonstrated. Here we describe a system that permits embodied prosthetic control; we show how monkeys (Macaca mulatta) use their motor cortical activity to control a mechanized arm replica in a self-feeding task. In addition to the three dimensions of movement, the subjects’ cortical signals also proportionally controlled a gripper on the end of the arm. Owing to the physical interaction between the monkey, the robotic arm and objects in the workspace, this new task presented a higher level of difficulty than previous virtual (cursor-control) experiments. Apart from an example of simple one-dimensional control23, previous experiments have lacked physical interaction even in cases where a robotic arm16,19,24 or hand20 was included in the control loop, because the subjects did not use it to interact with physical objects—an interaction that cannot be fully simulated. This demonstration of multi-degree-of-freedom embodied prosthetic control paves the way towards the development of dexterous prosthetic devices that could ultimately achieve arm and hand function at a near-natural level.

/pdf/cortical-control-of-a-prosthetic-arm-for-self-feeding-1ethevoxtu.pdf

Cortical control of a prosthetic arm for self-feeding

New silicon probes known as Neuropixels are shown to record from hundreds of neurons simultaneously in awake and freely moving rodents. Sensory, motor and cognitive operations involve the coordinated action of large neuronal populations across multiple brain regions. Existing technologies reliably measure activity from a relatively small number of neurons with high spatial and temporal resolution, or from a large volume of neurons with low resolution. Timothy Harris and colleagues describe the design, fabrication and performance of Neuropixels, a silicon probe that can measure well-isolated neural activity from hundreds of neurons. They integrated these probes into a lightweight system that could record activity simultaneously and with high fidelity from hundreds of neurons in awake and freely moving rodents. Sensory, motor and cognitive operations involve the coordinated action of large neuronal populations across multiple brain regions in both superficial and deep structures1,2. Existing extracellular probes record neural activity with excellent spatial and temporal (sub-millisecond) resolution, but from only a few dozen neurons per shank. Optical Ca2+ imaging3,4,5 offers more coverage but lacks the temporal resolution needed to distinguish individual spikes reliably and does not measure local field potentials. Until now, no technology compatible with use in unrestrained animals has combined high spatiotemporal resolution with large volume coverage. Here we design, fabricate and test a new silicon probe known as Neuropixels to meet this need. Each probe has 384 recording channels that can programmably address 960 complementary metal–oxide–semiconductor (CMOS) processing-compatible low-impedance TiN6 sites that tile a single 10-mm long, 70 × 20-μm cross-section shank. The 6 × 9-mm probe base is fabricated with the shank on a single chip. Voltage signals are filtered, amplified, multiplexed and digitized on the base, allowing the direct transmission of noise-free digital data from the probe. The combination of dense recording sites and high channel count yielded well-isolated spiking activity from hundreds of neurons per probe implanted in mice and rats. Using two probes, more than 700 well-isolated single neurons were recorded simultaneously from five brain structures in an awake mouse. The fully integrated functionality and small size of Neuropixels probes allowed large populations of neurons from several brain structures to be recorded in freely moving animals. This combination of high-performance electrode technology and scalable chip fabrication methods opens a path towards recording of brain-wide neural activity during behaviour.

/pdf/fully-integrated-silicon-probes-for-high-density-recording-nnob1hwgn8.pdf

Fully integrated silicon probes for high-density recording of neural activity

Prefrontal cortex is thought to have a fundamental role in flexible, context-dependent behaviour, but the exact nature of the computations underlying this role remains largely unknown. In particular, individual prefrontal neurons often generate remarkably complex responses that defy deep understanding of their contribution to behaviour. Here we study prefrontal cortex activity in macaque monkeys trained to flexibly select and integrate noisy sensory inputs towards a choice. We find that the observed complexity and functional roles of single neurons are readily understood in the framework of a dynamical process unfolding at the level of the population. The population dynamics can be reproduced by a trained recurrent neural network, which suggests a previously unknown mechanism for selection and integration of task-relevant inputs. This mechanism indicates that selection and integration are two aspects of a single dynamical process unfolding within the same prefrontal circuits, and potentially provides a novel, general framework for understanding context-dependent computations.

https://hearingbrain.org/docs/Mante_Newsome_2013.pdf

Context-dependent computation by recurrent dynamics in prefrontal cortex

Most theories of motor cortex have assumed that neural activity represents movement parameters. This view derives from what is known about primary visual cortex, where neural activity represents patterns of light. Yet it is unclear how well the analogy between motor and visual cortex holds. Single-neuron responses in motor cortex are complex, and there is marked disagreement regarding which movement parameters are represented. A better analogy might be with other motor systems, where a common principle is rhythmic neural activity. Here we find that motor cortex responses during reaching contain a brief but strong oscillatory component, something quite unexpected for a non-periodic behaviour. Oscillation amplitude and phase followed naturally from the preparatory state, suggesting a mechanistic role for preparatory neural activity. These results demonstrate an unexpected yet surprisingly simple structure in the population response. This underlying structure explains many of the confusing features of individual neural responses.

/pdf/neural-population-dynamics-during-reaching-1nuzsisoql.pdf

Neural population dynamics during reaching

https://users.ece.cmu.edu/~byronyu/papers/GolubCONB2016.pdf

Brain-computer interfaces for dissecting cognitive processes underlying sensorimotor control.

Spiking activity from neurophysiological experiments often exhibits dynamics beyond that driven by external stimulation, presumably reflecting the extensive recurrence of neural circuitry. Characterizing these dynamics may reveal important features of neural computation, particularly during internally-driven cognitive operations. For example, the activity of premotor cortex (PMd) neurons during an instructed delay period separating movement-target specification and a movement-initiation cue is believed to be involved in motor planning. We show that the dynamics underlying this activity can be captured by a low-dimensional non-linear dynamical systems model, with underlying recurrent structure and stochastic point-process output. We present and validate latent variable methods that simultaneously estimate the system parameters and the trial-by-trial dynamical trajectories. These methods are applied to characterize the dynamics in PMd data recorded from a chronically-implanted 96-electrode array while monkeys perform delayed-reach tasks.

/pdf/extracting-dynamical-structure-embedded-in-neural-activity-4wthtdr6ox.pdf

Extracting Dynamical Structure Embedded in Neural Activity

Simultaneous recordings of many neurons embedded within a recurrently-connected cortical network may provide concurrent views into the dynamical processes of that network, and thus its computational function. In principle, these dynamics might be identified by purely unsupervised, statistical means. Here, we show that a Hidden Switching Linear Dynamical Systems (HSLDS) model— in which multiple linear dynamical laws approximate a nonlinear and potentially non-stationary dynamical process—is able to distinguish different dynamical regimes within single-trial motor cortical activity associated with the preparation and initiation of hand movements. The regimes are identified without reference to behavioural or experimental epochs, but nonetheless transitions between them correlate strongly with external events whose timing may vary from trial to trial. The HSLDS model also performs better than recent comparable models in predicting the firing rate of an isolated neuron based on the firing rates of others, suggesting that it captures more of the "shared variance" of the data. Thus, the method is able to trace the dynamical processes underlying the coordinated evolution of network activity in a way that appears to reflect its computational role.

/pdf/dynamical-segmentation-of-single-trials-from-population-3s7wxps2ya.pdf

Dynamical segmentation of single trials from population neural data

Neural prostheses aim to improve the quality of life of severely disabled patients by translating neural activity into control signals for guiding prosthetic devices or computer cursors. We recently demonstrated that plan activity from premotor cortex, which specifies the endpoint of the upcoming arm movement, can be used to swiftly and accurately guide computer cursors to the desired target locations. However, these systems currently require additional, non-neural information to specify when plan activity is present. We report here the design and performance of state estimator algorithms for automatically detecting the presence of plan activity using neural activity alone. Prosthesis performance was nearly as good when state estimation was used as when perfect plan timing information was provided separately (~5 percentage points lower, when using 200 ms of plan activity). These results strongly suggest that a completely neurally-driven high-performance brain?computer interface is possible.

/pdf/free-paced-high-performance-brain-computer-interfaces-5eyg5v9qbv.pdf

Free-paced high-performance brain-computer interfaces.

https://www.cell.com/neuron/pdfExtended/S0896-6273(20)30565-1

Byron M. Yu

Papers

Brain-computer interfaces for dissecting cognitive processes underlying sensorimotor control.

Extracting Dynamical Structure Embedded in Neural Activity

Dynamical segmentation of single trials from population neural data

Free-paced high-performance brain-computer interfaces.

Slow Drift of Neural Activity as a Signature of Impulsivity in Macaque Visual and Prefrontal Cortex