Brain-Computer Interface (BCI), in essence, aims at controlling different assistive devices through the utilization of brain waves. It is worth noting that the application of BCI is not limited to medical applications, and hence, the research in this field has gained due attention. Moreover, the significant number of related publications over the past two decades further indicates the consistent improvements and breakthroughs that have been made in this particular field. Nonetheless, it is also worth mentioning that with these improvements, new challenges are constantly discovered. This article provides a comprehensive review of the state-of-the-art of a complete BCI system. First, a brief overview of electroencephalogram (EEG)-based BCI systems is given. Secondly, a considerable number of popular BCI applications are reviewed in terms of electrophysiological control signals, feature extraction, classification algorithms, and performance evaluation metrics. Finally, the challenges to the recent BCI systems are discussed, and possible solutions to mitigate the issues are recommended.

/pdf/current-status-challenges-and-possible-solutions-of-eeg-4el4wek5td.pdf

Current Status, Challenges, and Possible Solutions of EEG-Based Brain-Computer Interface: A Comprehensive Review

Brain-machine interfaces (BMIs) can be used to decode brain activity into commands to control external devices. This paper presents the decoding of intuitive upper extremity imagery for multi-directional arm reaching tasks in three-dimensional (3D) environments. We designed and implemented an experimental environment in which electroencephalogram (EEG) signals can be acquired for movement execution and imagery. Fifteen subjects participated in our experiments. We proposed a multi-directional convolution neural network-bidirectional long short-term memory network (MDCBN)-based deep learning framework. The decoding performances for six directions in 3D space were measured by the correlation coefficient (CC) and the normalized root mean square error (NRMSE) between predicted and baseline velocity profiles. The grand-averaged CCs of multi-direction were 0.47 and 0.45 for the execution and imagery sessions, respectively, across all subjects. The NRMSE values were below 0.2 for both sessions. Furthermore, in this study, the proposed MDCBN was evaluated by two online experiments for real-time robotic arm control, and the grand-averaged success rates were approximately 0.60 (±0.14) and 0.43 (±0.09), respectively. Hence, we demonstrate the feasibility of intuitive robotic arm control based on EEG signals for real-world environments.

Brain-Controlled Robotic Arm System Based on Multi-Directional CNN-BiLSTM Network Using EEG Signals

Deep learning for motor imagery EEG-based classification: A review

Since the launch of the first consumer grade EEG measuring sensors ‘NeuroSky Mindset’ in 2007, the market has witnessed an introduction of at least one new product every year by competing manufacturers, which include NeuroSky, Emotiv, interaXon and OpenBCI. There are numerous variations in the make and versions, but these products clearly share the key selling points of affordability, portability, and ease of use. These features are patently well placed provided one of the main objectives for their development is to attract a new target group of commercial users. Nevertheless, with several decades of traditional EEG usage in clinical and experimental settings, the shift toward commercial and engineering sides has not been achieved without skepticism. With this in mind, researchers in related fields have been tirelessly working to ensure that these putatively novel features were not introduced at the expense of efficiency and accuracy by conducting validation studies to compare the performance of data derived from consumer grade EEG devices with ones from standard research grade counterparts. In this review, we seek to provide the detail of the products supplied by the major players, summarize studies that evaluate consumer product’s performance against research grade devices, the key areas of applications that these consumer grade devices have been employed in over the past five years or so, and finally give our perspectives on the limitations and what these innovative tools could offer going forward in terms of research and commercial applications.

Consumer Grade EEG Measuring Sensors as Research Tools: A Review

Objective Learning the structures and unknown correlations of a motor imagery electroencephalogram (MI-EEG) signal is important for its classification. It is also a major challenge to obtain good classification accuracy from the increased number of classes and increased variability from different people. In this study, a four-class MI task is investigated. Approach An end-to-end novel hybrid deep learning scheme is developed to decode the MI task from EEG data. The proposed algorithm consists of two parts: a. A one-versus-rest filter bank common spatial pattern is adopted to preprocess and pre-extract the features of the four-class MI signal. b. A hybrid deep network based on the convolutional neural network and long-term short-term memory network is proposed to extract and learn the spatial and temporal features of the MI signal simultaneously. Main results The main contribution of this paper is to propose a hybrid deep network framework to improve the classification accuracy of the four-class MI-EEG signal. The hybrid deep network is a subject-independent shared neural network, which means it can be trained by using the training data from all subjects to form one model. Significance The classification performance obtained by the proposed algorithm on brain-computer interface (BCI) competition IV dataset 2a in terms of accuracy is 83% and Cohen's kappa value is 0.80. Finally, the shared hybrid deep network is evaluated by every subject respectively, and the experimental results illustrate that the shared neural network has satisfactory accuracy. Thus, the proposed algorithm could be of great interest for real-life BCIs.

https://iopscience.iop.org/article/10.1088/1741-2552/ab3471/pdf

A novel hybrid deep learning scheme for four-class motor imagery classification.

Feature extraction and classification play an important role in brain–computer interface (BCI) systems. In traditional approaches, methods in pattern recognition field are adopted to solve these problems. Nowadays, the deep learning theory has developed so fast that researchers have employed it in many areas like computer vision and speech recognition, which have achieved remarkable results. However, few people introduce the deep learning method into the study of biomedical signals, especially EEG signals. In this paper, a wavelet transform-based input, which combines the time-frequency images of C3, Cz, and C4 channels, is proposed to extract the feature of motor imagery EEG signal. Then, a 2-Layer convolutional neural network is built as the classifier and convolutional kernels of different sizes are validated. The performance obtained by the proposed approach is evaluated by accuracy and Kappa value. The accuracy on dataset III from BCI competition II reaches 90%, and the best Kappa value on dataset 2a from competition IV is greater than many of other methods. In addition, the proposed method utilizes a resized small input, which reduces calculation complexity, so the training period is relatively faster. The results show that the method using convolutional neural network can be comparable or better than the other state-of-the-art approaches, and the performance will be improved when there is sufficient data.

Wavelet Transform Time-Frequency Image and Convolutional Network-Based Motor Imagery EEG Classification

Brain computer interface (BCI) adopts human brain signals to control external devices directly without using normal neural pathway. Recent study has explored many applications, such as controlling a teleoperation robot by electroencephalography (EEG) signals. However, utilizing the motor imagery EEG-based BCI to perform teleoperation for reach and grasp task still has many difficulties, especially in continuous multidimensional control of robot and tactile feedback. In this research, a motor imagery EEG-based continuous teleoperation robot control system with tactile feedback was proposed. Firstly, mental imagination of different hand movements was translated into continuous command to control the remote robotic arm to reach the hover area of the target through a wireless local area network (LAN). Then, the robotic arm automatically completed the task of grasping the target. Meanwhile, the tactile information of remote robotic gripper was detected and converted to the feedback command. Finally, the vibrotactile stimulus was supplied to users to improve their telepresence. Experimental results demonstrate the feasibility of using the motor imagery EEG acquired by wireless portable equipment to realize the continuous teleoperation robot control system to finish the reach and grasp task. The average two-dimensional continuous control success rates for online Task 1 and Task 2 of the six subjects were 78.0% ± 6.1% and 66.2% ± 6.0%, respectively. Furthermore, compared with the traditional EEG triggered robot control using the predefined trajectory, the continuous fully two-dimensional control can not only improve the teleoperation robot system’s efficiency but also give the subject a more natural control which is critical to human–machine interaction (HMI). In addition, vibrotactile stimulus can improve the operator’s telepresence and task performance.

https://www.mdpi.com/2079-9292/9/1/174/pdf

Motor Imagery Based Continuous Teleoperation Robot Control with Tactile Feedback

The classification of motor imagery Electroencephalogram (EEG) of the same limb is important for natural control of neuroprosthesis. Due to the close spatial representations on the motor cortex area of the brain, the discrimination of the different motor imagery tasks is challenging. In this paper, phase synchronization information was proposed to classify motor imagery EEG within the same limb. In addition, non-portable was compared with portable EEG acquisition equipment for the purpose of making the brain computer interface (BCI) system more practical. In the non-portable case, the average accuracy of the binary classification and 3-class classification was 60.6% and 42.7%. In the portable case, the average EEG decoding accuracy of 58.5% and 39.9% was achieved for the two and three tasks. Furthermore, in both two cases, different sets of electrode pairs got the similar results. Moreover, we found that the proposed phase information based method was less sensitive to the number of EEG channels and had less performance degradation in portable EEG equipment. These results show it is possible to use phase synchronization information to discriminate different motor imagery tasks within the same limb. Eventually, this will potentially make the control of neuroprosthesis or other rehabilitation device more natural and intuitive.

/pdf/phase-synchronization-information-for-classifying-motor-21nmokr1zi.pdf

Phase Synchronization Information for Classifying Motor Imagery EEG From the Same Limb

The invention discloses a teleoperation robot system based on hybrid bioelectric signal driving. The teleoperation robot system comprises an electroencephalogram signal acquisition module, an electro-oculogram signal acquisition module, a signal processing module, a wireless network transmission module, a robot control module, a force tactile detection module, a visual detection module, a tactilefeedback module and a visual feedback module. The invention further discloses a control method of the remote operation robot system based on hybrid bioelectric signal driving, the remote operation robot is controlled by combining the motor imagery electroencephalogram signals of an operator and the eye electric signals, meanwhile, the control state is fed back in real time through visual feedbackand force tactile feedback, and the control efficiency and precision are improved.

teleoperation robot system and method based on hybrid bioelectric signal driving

The invention discloses a rehabilitation auxiliary robot driven by a hybrid brain-computer interface. The robot comprises a data collection module, a PC processing module, a mechanical arm control module, a mechanical arm and a feedback module. The data collection module is used for collecting sight line movement information and motor imagery electroencephalogram signals and collecting depth imageinformation a user face and a target object, the PC processing module is used for obtaining the target object which the user wants to grab according to the motor imagery electroencephalogram signalsof the user and extracting position information according to the received image information, the mechanical arm control module is used for planning the movement path in which the mechanical arm conveys the target object to the user face according the position information of the user face and the target object, and producing mechanical arm control signals by synthesizing the sight line movement information of the user and the planned movement path so as to control movement of the mechanical arm, the mechanical arm is used for moving under the control of the mechanical arm control module, and the feedback module is installed on the mechanical arm and used for feeding back signals representing whether the target is successfully clamped. The rehabilitation auxiliary robot synthesizes characteristics of the brain-computer interface technology and the characteristics of the rehabilitation robot technology, and is more beneficial to paint rehabilitation training.

Li Wenlong

Papers

Wavelet Transform Time-Frequency Image and Convolutional Network-Based Motor Imagery EEG Classification

Motor Imagery Based Continuous Teleoperation Robot Control with Tactile Feedback

Phase Synchronization Information for Classifying Motor Imagery EEG From the Same Limb

teleoperation robot system and method based on hybrid bioelectric signal driving

Rehabilitation auxiliary robot driven by hybrid brain-computer interface