During object manipulation tasks, the brain selects and implements action-phase controllers that use sensory predictions and afferent signals to tailor motor output to the physical properties of the objects involved. Analysis of signals in tactile afferent neurons and central processes in humans reveals how contact events are encoded and used to monitor and update task performance.

Coding and use of tactile signals from the fingertips in object manipulation tasks

The exploits of Martina Navratilova and Roger Federer represent the pinnacle of motor learning. However, when considering the range and complexity of the processes that are involved in motor learning, even the mere mortals among us exhibit abilities that are impressive. We exercise these abilities when taking up new activities — whether it is snowboarding or ballroom dancing — but also engage in substantial motor learning on a daily basis as we adapt to changes in our environment, manipulate new objects and refine existing skills. Here we review recent research in human motor learning with an emphasis on the computational mechanisms that are involved.

/pdf/principles-of-sensorimotor-learning-59h00bcrss.pdf

Principles of sensorimotor learning.

Hand loss is a highly disabling event that markedly affects the quality of life. To achieve a close to natural replacement for the lost hand, the user should be provided with the rich sensations that we naturally perceive when grasping or manipulating an object. Ideal bidirectional hand prostheses should involve both a reliable decoding of the user's intentions and the delivery of nearly "natural" sensory feedback through remnant afferent pathways, simultaneously and in real time. However, current hand prostheses fail to achieve these requirements, particularly because they lack any sensory feedback. We show that by stimulating the median and ulnar nerve fascicles using transversal multichannel intrafascicular electrodes, according to the information provided by the artificial sensors from a hand prosthesis, physiologically appropriate (near-natural) sensory information can be provided to an amputee during the real-time decoding of different grasping tasks to control a dexterous hand prosthesis. This feedback enabled the participant to effectively modulate the grasping force of the prosthesis with no visual or auditory feedback. Three different force levels were distinguished and consistently used by the subject. The results also demonstrate that a high complexity of perception can be obtained, allowing the subject to identify the stiffness and shape of three different objects by exploiting different characteristics of the elicited sensations. This approach could improve the efficacy and "life-like" quality of hand prostheses, resulting in a keystone strategy for the near-natural replacement of missing hands.

/pdf/restoring-natural-sensory-feedback-in-real-time-48ihx8sc2g.pdf

Restoring Natural Sensory Feedback in Real-Time Bidirectional Hand Prostheses

Objective. In a previous study we demonstrated continuous translation, orientation and one-dimensional grasping control of a prosthetic limb (seven degrees of freedom) by a human subject with tetraplegia using a brain?machine interface (BMI). The current study, in the same subject, immediately followed the previous work and expanded the scope of the control signal by also extracting hand-shape commands from the two 96-channel intracortical electrode arrays implanted in the subject?s left motor cortex. Approach. Four new control signals, dictating prosthetic hand shape, replaced the one-dimensional grasping in the previous study, allowing the subject to control the prosthetic limb with ten degrees of freedom (three-dimensional (3D) translation, 3D orientation, four-dimensional hand shaping) simultaneously. Main results. Robust neural tuning to hand shaping was found, leading to ten-dimensional (10D) performance well above chance levels in all tests. Neural unit preferred directions were broadly distributed through the 10D space, with the majority of units significantly tuned to all ten dimensions, instead of being restricted to isolated domains (e.g. translation, orientation or hand shape). The addition of hand shaping emphasized object-interaction behavior. A fundamental component of BMIs is the calibration used to associate neural activity to intended movement. We found that the presence of an object during calibration enhanced successful shaping of the prosthetic hand as it closed around the object during grasping. Significance. Our results show that individual motor cortical neurons encode many parameters of movement, that object interaction is an important factor when extracting these signals, and that high-dimensional operation of prosthetic devices can be achieved with simple decoding algorithms. ClinicalTrials.gov Identifier: NCT01364480.

https://iopscience.iop.org/article/10.1088/1741-2560/12/1/016011/pdf

Ten-dimensional anthropomorphic arm control in a human brain?machine interface: difficulties, solutions, and limitations

https://www.cell.com/trends/cognitive-sciences/pdf/S1364-6613(08)00232-5.pdf

Trans-saccadic perception

https://www.jsmf.org/meetings/2008/may/Flanagan%20et%20al%20Curr%20Opin%20Neurobiol%202006.pdf

Control strategies in object manipulation tasks.

Most object manipulation tasks involve a series of actions demarcated by mechanical contact events, and gaze is typically directed to the locations of these events as the task unfolds. Here, we examined the timing of gaze shifts relative to hand movements in a task in which participants used a handle to contact sequentially five virtual objects located in a horizontal plane. This task was performed both with and without visual feedback of the handle position. We were primarily interested in whether gaze shifts, which in our task shifted from a given object to the next about 100 ms after contact, were predictive or triggered by tactile feedback related to contact. To examine this issue, we included occasional catch contacts where forces simulating contact between the handle and object were removed. In most cases, removing force did not alter the timing of gaze shifts irrespective of whether or not vision of handle position was present. However, in about 30% of the catch contacts, gaze shifts were delayed. This percentage corresponded to the fraction of contacts with force feedback in which gaze shifted more than 130 ms after contact. We conclude that gaze shifts are predictively controlled but timed so that the hand actions around the time of contact are captured in central vision. Furthermore, a mismatch between the expected and actual tactile information related to the contact can lead to a reorganization of gaze behavior for gaze shifts executed greater than 130 ms after a contact event.

/pdf/eye-hand-coordination-in-a-sequential-target-contact-task-rl5n5bqosd.pdf

Eye–hand coordination in a sequential target contact task

Most object manipulation tasks involve a series of actions demarcated by mechanical contact events, and gaze is typically directed to the locations of these events as the task unfolds. Here, we examined the timing of gaze shifts relative to hand movements in a task in which participants used a handle to contact sequentially Wve virtual objects located in a horizontal plane. This task was performed both with and without visual feedback of the handle position. We were primarily interested in whether gaze shifts, which in our task shifted from a given object to the next about 100 ms after contact, were predictive or triggered by tactile feedback related to contact. To examine this issue, we included occasional catch contacts where forces simulating contact between the handle and object were removed. In most cases, removing force did not alter the timing of gaze shifts irrespective of whether or not vision of handle position was present. However, in about 30% of the catch contacts, gaze shifts were delayed. This percentage corresponded to the fraction of contacts with force feedback in which gaze shifted more than 130 ms after contact. We conclude that gaze shifts are predictively controlled but timed so that the hand actions around the time of contact are captured in central vision. Furthermore, a mismatch between the expected and actual tactile information related to the contact can lead to a reorganization of gaze behavior for gaze shifts executed greater than 130 ms after a contact event.

/pdf/erratum-to-eye-hand-coordination-in-a-sequential-target-25h9pxn9co.pdf

Miles C. Bowman

Papers

Control strategies in object manipulation tasks.

Eye–hand coordination in a sequential target contact task

Erratum to: Eye–hand coordination in a sequential target contact task