While brain-computer interfaces (BCIs) can provide communication to people who are locked-in, they suffer from a very low information transfer rate. Further, using a BCI requires a concentration effort and using it continuously can be tiring. The brain controlled wheelchair (BCW) described in this paper aims at providing mobility to BCI users despite these limitations, in a safe and efficient way. Using a slow but reliable P300 based BCI, the user selects a destination amongst a list of predefined locations. While the wheelchair moves on virtual guiding paths ensuring smooth, safe, and predictable trajectories, the user can stop the wheelchair by using a faster BCI. Experiments with nondisabled subjects demonstrated the efficiency of this strategy. Brain control was not affected when the wheelchair was in motion, and the BCW enabled the users to move to various locations in less time and with significantly less control effort than other control strategies proposed in the literature.

/pdf/a-brain-controlled-wheelchair-to-navigate-in-familiar-5cre1p8c4w.pdf

A Brain Controlled Wheelchair to Navigate in Familiar Environments

Foundations of Robotics presents the fundamental concepts and methodologies for the analysis, design, and control of robot manipulators It explains the physical meaning of the concepts and equations used, and it provides, in an intuitively clear way, the necessary background in kinetics, linear algebra, and control theory Illustrative examples appear throughoutThe author begins by discussing typical robot manipulator mechanisms and their controllers He then devotes three chapters to the analysis of robot manipulator mechanisms He covers the kinematics of robot manipulators, describing the motion of manipulator links and objects related to manipulation A chapter on dynamics includes the derivation of the dynamic equations of motion, their use for control and simulation and the identification of inertial parameters The final chapter develops the concept of manipulabilityThe second half focuses on the control of robot manipulators Various position-control algorithms that guide the manipulator's end effector along a desired trajectory are described Two typical methods used to control the contact force between the end effector and its environments are detailed For manipulators with redundant degrees of freedom, a technique to develop control algorithms for active utilization of the redundancy is described Appendixes give compact reviews of the function atan2, pseudo inverses, singular-value decomposition, and Lyapunov stability theoryTsuneo Yoshikawa teaches in the Division of Applied Systems Science in Kyoto University's Faculty of Engineering

Foundations of robotics : analysis and control

EEG-based brain-controlled mobile robots can serve as powerful aids for severely disabled people in their daily life, especially to help them move voluntarily. In this paper, we provide a comprehensive review of the complete systems, key techniques, and evaluation issues of brain-controlled mobile robots along with some insights into related future research and development issues. We first review and classify various complete systems of brain-controlled mobile robots into two categories from the perspective of their operational modes. We then describe key techniques that are used in these brain-controlled mobile robots including the brain-computer interface techniques and shared control techniques. This description is followed by an analysis of the evaluation issues of brain-controlled mobile robots including participants, tasks and environments, and evaluation metrics. We conclude this paper with a discussion of the current challenges and future research directions.

EEG-Based Brain-Controlled Mobile Robots: A Survey

Robotic assistive devices are used increasingly to improve the independence and quality of life of persons with disabilities. Devices as varied as robotic feeders, smart-powered wheelchairs, independent mobile robots, and socially assistive robots are becoming more clinically relevant. There is a growing importance for the rehabilitation professional to be aware of available systems and ongoing research efforts. The aim of this article is to describe the advances in assistive robotics that are relevant to professionals serving persons with disabilities. This review breaks down relevant advances into categories of Assistive Robotic Systems, User Interfaces and Control Systems, Sensory and Feedback Systems, and User Perspectives. An understanding of the direction that assistive robotics is taking is important for the clinician and researcher alike; this review is intended to address this need.

The role of assistive robotics in the lives of persons with disability.

To increase the ability of brain-machine interfaces (BMIs) to control advanced prostheses such as the modular prosthetic limb (MPL), we are developing a novel system: the Hybrid Augmented Reality Multimodal Operation Neural Integration Environment (HARMONIE). This system utilizes hybrid input, supervisory control, and intelligent robotics to allow users to identify an object (via eye tracking and computer vision) and initiate (via brain-control) a semi-autonomous reach-grasp-and-drop of the object by the MPL. Sequential iterations of HARMONIE were tested in two pilot subjects implanted with electrocorticographic (ECoG) and depth electrodes within motor areas. The subjects performed the complex task in 71.4% (20/28) and 67.7% (21/31) of trials after minimal training. Balanced accuracy for detecting movements was 91.1% and 92.9%, significantly greater than chance accuracies (p < 0.05). After BMI-based initiation, the MPL completed the entire task 100% (one object) and 70% (three objects) of the time. The MPL took approximately 12.2 s for task completion after system improvements implemented for the second subject. Our hybrid-BMI design prevented all but one baseline false positive from initiating the system. The novel approach demonstrated in this proof-of-principle study, using hybrid input, supervisory control, and intelligent robotics, addresses limitations of current BMIs.

Demonstration of a Semi-Autonomous Hybrid Brain–Machine Interface Using Human Intracranial EEG, Eye Tracking, and Computer Vision to Control a Robotic Upper Limb Prosthetic

A wheelchair-mounted robotic arm (WMRA) system was designed and built to meet the needs of mobilityimpaired persons with limitations of upper extremities, and to exceed the capabilities of current devices of this type. The control of this 9-degree-of-freedom system expands upon conventional control methods and combines the 7-DoF robotic arm control with the 2-degree-of-freedom power wheelchair control. The 3- degrees of redundancy are optimized to effectively perform activities of daily living and overcome singularities, joint limits and some workspace limitations. The control system is designed for teleoperated or autonomous coordinated Cartesian control, which offers expandability for future research. A P300 Brain Computer Interface (BCI), the BCI2000, was implemented to control the WMRA system. The control is done by recording and analysing the brain activity through an electrode cap while providing visual stimulation to the user via a visual matrix. The visual matrix contains a symbolic or an alphabetic array corresponding to the motion of the WMRA. By recognizing online and in real-time, which element in the matrix elicited a P300, the BCI system can identify which element the user chose to communicate. The chosen element is then communicated to the controller of the WMRA system. The speed and accuracy of the BCI system was tested. This paper gives details of the WMRA's integration with the BCI2000 and documents the experimental results of the BCI and the WMRA in simulation.

Control of a 9-DoF Wheelchair-mounted robotic arm system using a P300 Brain Computer Interface: Initial experiments

This invention provides a compact self-contained design of a wrist device for an electrically powered prosthesis. The wrist prosthesis, which provides user controlled motion in two directions, flexion/extension and rotation. Through the unique mounting of the one or more actuators or motors, a significant space savings is achieved. The modular terminal device distal mounting platform and prosthesis proximal mounting structure allow for this wrist device to function with many commercially available terminal devices and prosthesis designs. In a more particular embodiment related to a prosthesis the wrist device utilizes a system of gears, which achieves appropriately externally powered motion and is configured to provide wrist flexion as well as rotation. The prosthesis allows user controlled flexion/extension and rotation by providing two actuating motors and a series of gears, which provides externally powered flexion, extension, and rotation in various ranges.

Joint Prosthetic Device

Three decades ago, Farwell and Donchin [1] developed a computer system based on the method of electroencephalography (EEG) that enables individuals to communicate with their environment without using any neuromuscular function. This P300 BCI speller makes use of the well-studied observation that the brain reacts differently to different stimuli, based on the level of attention given to the stimulus and the specific processing triggered by the stimulus. Since this first report in 1988, several brain-computer interface (BCI) systems have been developed and constantly improved. We have previously demonstrated that the P300-BCI can control a wheelchair-mounted robotic arm (WMRA) system [2].Copyright © 2008 by ASME

Implementation of a P-300 Brain Computer Interface for the Control of a Wheelchair Mounted Robotic Arm System

Biological studies have contributed to the control of mobile robots. Studies from the field of neuroscience to the field of psychology have shown roboticists different ways to control mechanical devices. A number of studies pertaining to human reaching and grasping have been done in the past few decades. These studies provide the psychological and neuropsychological evidence regarding human perception and movement of the human hand. This research aims to incorporate ideas from biological studies to control a 9-DoF Wheelchair-Mounted Robotic Arm (WMRA) system to be used as an assistive device. This paper gives succinct details of the biological studies done related to human sensorimotor actions and applies those ideas to control the WMRA. These ideas are incorporated into the WMRA system to work within its limits in its intended environment as an assistive device. This paper provides the implementation and initial simulation results.

Using biological approaches for the control of a 9-DoF wheelchair-mounted robotic arm system: Initial experiments

This paper describes the design and implementation of a control system that integrates visual and haptic information to give assistive force feedback through a haptic controller (Omni Phantom) to the user. A sensor-based assistive function and velocity scaling program provides force feedback that helps the user complete trajectory following exercises for rehabilitation purposes. This system also incorporates a PUMA robot for teleoperation, which implements a camera and a laser range finder, controlled in real time by a PC, were implemented into the system to help the user to define the intended path to the selected target. The real-time force feedback from the remote robot to the haptic controller is made possible by using effective multithreading programming strategies in the control system design and by novel sensor integration. The sensor-based assistant function concept applied to teleoperation as well as shared control enhances the motion range and manipulation capabilities of the users executing rehabilitation exercises such as trajectory following along a sensor-based defined path. The system is modularly designed to allow for integration of different master devices and sensors. Furthermore, because this real-time system is versatile the haptic component can be used separately from the telerobotic component; in other words, one can use the haptic device for rehabilitation purposes for cases in which assistance is needed to perform tasks (e.g., stroke rehab) and also for teleoperation with force feedback and sensor assistance in either supervisory or automatic modes.

Kathryn J. De Laurentis

Papers

Control of a 9-DoF Wheelchair-mounted robotic arm system using a P300 Brain Computer Interface: Initial experiments

Joint Prosthetic Device

Implementation of a P-300 Brain Computer Interface for the Control of a Wheelchair Mounted Robotic Arm System

Using biological approaches for the control of a 9-DoF wheelchair-mounted robotic arm system: Initial experiments

Design and implementation of visual-haptic assistive control system for virtual rehabilitation exercise and teleoperation manipulation