For an individual with tetraplegia assistive robotic arms provide a potentially invaluable opportunity for rehabilitation. However, there is a lack of available control methods to allow these individuals to fully control the assistive arms. Here we show that it is possible for an individual with tetraplegia to use the tongue to fully control all 14 movements of an assistive robotic arm in a three dimensional space using a wireless intraoral control system, thus allowing for numerous activities of daily living. We developed a tongue-based robotic control method incorporating a multi-sensor inductive tongue interface. One abled-bodied individual and one individual with tetraplegia performed a proof of concept study by controlling the robot with their tongue using direct actuator control and endpoint control, respectively. After 30 min of training, the able-bodied experimental participant tongue controlled the assistive robot to pick up a roll of tape in 80% of the attempts. Further, the individual with tetraplegia succeeded in fully tongue controlling the assistive robot to reach for and touch a roll of tape in 100% of the attempts and to pick up the roll in 50% of the attempts. Furthermore, she controlled the robot to grasp a bottle of water and pour its contents into a cup; her first functional action in 19 years. To our knowledge, this is the first time that an individual with tetraplegia has been able to fully control an assistive robotic arm using a wireless intraoral tongue interface. The tongue interface used to control the robot is currently available for control of computers and of powered wheelchairs, and the robot employed in this study is also commercially available. Therefore, the presented results may translate into available solutions within reasonable time.

/pdf/wireless-intraoral-tongue-control-of-an-assistive-robotic-pyrv6dio4a.pdf

Wireless intraoral tongue control of an assistive robotic arm for individuals with tetraplegia

For severely paralyzed individuals, alternative computer interfaces are becoming increasingly essential for everyday life as social and vocational activities are facilitated by information technology and as the environment becomes more automatic and remotely controllable Tongue computer interfaces have proven to be desirable by the users partly due to their high degree of aesthetic acceptability, but so far the mature systems have shown a relatively low error-free text typing efficiency This paper evaluated the intra-oral inductive tongue computer interface (ITCI) in its intended use: Error-free text typing in a generally available text editing system, Word Individuals with tetraplegia and able bodied individuals used the ITCI for typing using a MATLAB interface and for Word typing for 4 to 5 experimental days, and the results showed an average error-free text typing rate in Word of 116 correct characters/min across all participants and of 155 correct characters/min for participants familiar with tongue piercings Improvements in typing rates between the sessions suggest that typing ratescan be improved further through long-term use of the ITCI

Error-Free Text Typing Performance of an Inductive Intra-Oral Tongue Computer Interface for Severely Disabled Individuals

This paper explores a new wearable system, called TYTH, that enables a novel form of human computer interaction based on the relative location and interaction between the user's tongue and teeth. TYTH allows its user to interact with a computing system by tapping on their teeth. This form of interaction is analogous to using a finger to type on a keypad except that the tongue substitutes for the finger and the teeth for the keyboard. We study the neurological and anatomical structures of the tongue to design TYTH so that the obtrusiveness and social awkwardness caused by the wearable is minimized while maximizing its accuracy and sensing sensitivity. From behind the user's ears, TYTH senses the brain signals and muscle signals that control tongue movement sent from the brain and captures the miniature skin surface deformation caused by tongue movement. We model the relationship between tongue movement and the signals recorded, from which a tongue localization technique and tongue-teeth tapping detection technique are derived. Through a prototyping implementation and an evaluation with 15 subjects, we show that TYTH can be used as a form of hands-free human computer interaction with 88.61% detection rate and promising adoption rate by users.

TYTH-Typing On Your Teeth: Tongue-Teeth Localization for Human-Computer Interface

This paper presents a low-power stand-alone tongue drive system (sTDS) used for individuals with severe disabilities to potentially control their environment such as computer, smartphone, and wheelchair using their voluntary tongue movements. A low-power local processor is proposed, which can perform signal processing to convert raw magnetic sensor signals to user-defined commands, on the sTDS wearable headset, rather than sending all raw data out to a PC or smartphone. The proposed sTDS significantly reduces the transmitter power consumption and subsequently increases the battery life. Assuming the sTDS user issues one command every 20 ms, the proposed local processor reduces the data volume that needs to be wirelessly transmitted by a factor of 64, from 9.6 to 0.15 kb/s. The proposed processor consists of three main blocks: serial peripheral interface bus for receiving raw data from magnetic sensors, external magnetic interference attenuation to attenuate external magnetic field from the raw magnetic signal, and a machine learning classifier for command detection. A proof-of-concept prototype sTDS has been implemented with a low-power IGLOO-nano field programmable gate array (FPGA), bluetooth low energy, battery and magnetic sensors on a headset, and tested. At clock frequency of 20 MHz, the processor takes 6.6 μs and consumes 27 nJ for detecting a command with a detection accuracy of 96.9%. To further reduce power consumption, an application-specified integrated circuit processor for the sTDS is implemented at the postlayout level in 65-nm CMOS technology with 1-V power supply, and it consumes 0.43 mW, which is 10× lower than FPGA power consumption and occupies an area of only 0.016 mm
 2
.

A Low-Power Wearable Stand-Alone Tongue Drive System for People With Severe Disabilities

Multimodal Tongue Drive System (mTDS) is a highly integrated wireless assistive technology (AT) in the form of a lightweight wearable headset that utilizes three remaining key control and communication abilities in people with severe physical disabilities, such as tetraplegia, to provide them with effective access to computers: 1) tongue motion for discrete/switch-based control (e.g., clicking), 2) head tracking for proportional control (e.g., mouse pointer movements), and 3) speech recognition for typing, all available simultaneously. The mTDS architecture is presented here with new sensor signal processing algorithm for head tracking. To evaluate the device performance, it was compared against keyboard-and-mouse (KnM) combination, the gold standard in computer input methods, by 15 able-bodied participants, who used both mTDS and KnM to generate and sent an email with randomly selected content, under a 5-minute time constraint. In four repetitions, in the last trial, it took participants only 1.8 times longer to complete the email task, on average, using the mTDS versus KnM at 82.4% typing accuracy. Mean task completion time and typing accuracy improved 24.6% and 18.8% from first to fourth trial using mTDS. Multimodal simultaneous discrete and proportional control input options of mTDS, plus rapid typing, is expected to provide more effective computer access to people with severe physical disabilities.

Simultaneous Multimodal PC Access for People With Disabilities by Integrating Head Tracking, Speech Recognition, and Tongue Motion

Purpose: Individuals with tetraplegia depend on alternative interfaces in order to control computers and other electronic equipment. Current interfaces are often limited in the number of available control commands, and may compromise the social identity of an individual due to their undesirable appearance. The purpose of this study was to implement an alternative computer interface, which was fully embedded into the oral cavity and which provided multiple control commands.Methods: The development of a wireless, intraoral, inductive tongue computer was described. The interface encompassed a 10-key keypad area and a mouse pad area. This system was embedded wirelessly into the oral cavity of the user. The functionality of the system was demonstrated in two tetraplegic individuals and two able-bodied individualsResults: The system was invisible during use and allowed the user to type on a computer using either the keypad area or the mouse pad. The maximal typing rate was 1.8 s for repetitively typing ...

/pdf/development-and-functional-demonstration-of-a-wireless-3ooqeciqyp.pdf

Development and functional demonstration of a wireless intraoral inductive tongue computer interface for severely disabled persons

Autonomous mobile robot is usually equipped a finite amount of energy, it can accordingly work in finite time. Inspired from swarm behavior of collecting and sharing food of natural animals, e.g. honey-bee, game theory and mobile agents in multi-agent systems we issue a novel approach to distribute energy for multi-task autonomous mobile robots in order to keep group of mobile robot long-lived. In this paper we describe practical deployment of a group of mobile robot with possibility of carrying and then exchanging fuel cell, e.g. battery to other robot. The battery exchange solution is basically based on probabilistic model of total energy of each robot in its local vicinity. Early implementation that includes modular hardware and control architecture to demonstrate possibility of the approach is presented.

Henrik Schioeler

Papers

Development and functional demonstration of a wireless intraoral inductive tongue computer interface for severely disabled persons

Probabilistic Distributed Energy for Long-lived Group of Mobile Robots