Objective Electroencephalography (EEG) analysis has been an important tool in neuroscience with applications in neuroscience, neural engineering (e.g. Brain-computer interfaces, BCI's), and even commercial applications. Many of the analytical tools used in EEG studies have used machine learning to uncover relevant information for neural classification and neuroimaging. Recently, the availability of large EEG data sets and advances in machine learning have both led to the deployment of deep learning architectures, especially in the analysis of EEG signals and in understanding the information it may contain for brain functionality. The robust automatic classification of these signals is an important step towards making the use of EEG more practical in many applications and less reliant on trained professionals. Towards this goal, a systematic review of the literature on deep learning applications to EEG classification was performed to address the following critical questions: (1) Which EEG classification tasks have been explored with deep learning? (2) What input formulations have been used for training the deep networks? (3) Are there specific deep learning network structures suitable for specific types of tasks? Approach A systematic literature review of EEG classification using deep learning was performed on Web of Science and PubMed databases, resulting in 90 identified studies. Those studies were analyzed based on type of task, EEG preprocessing methods, input type, and deep learning architecture. Main results For EEG classification tasks, convolutional neural networks, recurrent neural networks, deep belief networks outperform stacked auto-encoders and multi-layer perceptron neural networks in classification accuracy. The tasks that used deep learning fell into five general groups: emotion recognition, motor imagery, mental workload, seizure detection, event related potential detection, and sleep scoring. For each type of task, we describe the specific input formulation, major characteristics, and end classifier recommendations found through this review. Significance This review summarizes the current practices and performance outcomes in the use of deep learning for EEG classification. Practical suggestions on the selection of many hyperparameters are provided in the hope that they will promote or guide the deployment of deep learning to EEG datasets in future research.

https://iopscience.iop.org/article/10.1088/1741-2552/ab0ab5/pdf

Deep learning for electroencephalogram (EEG) classification tasks: a review.

Brain–computer interface (BCI) is a system empowering humans to communicate with or control the outside world with exclusively brain intentions. Electroencephalography (EEG)-based BCI is one of the promising solutions due to its convenient and portable instruments. Despite the extensive research of EEG in recent years, it is still challenging to interpret EEG signals effectively due to its nature of noise and difficulties in capturing the inconspicuous relations between EEG signals and specific brain activities. Most existing works either only consider EEG as chain-like sequences while neglecting complex dependencies between adjacent signals or requiring complex preprocessing. In this paper, we introduce two deep learning-based frameworks with novel spatio-temporal preserving representations of raw EEG streams to precisely identify human intentions. The two frameworks consist of both convolutional and recurrent neural networks effectively exploring the preserved spatial and temporal information in either a cascade or a parallel manner. Extensive experiments on a large scale movement intention EEG dataset (108 subjects, 3 145 160 EEG records) have demonstrated that the proposed frameworks achieve high accuracy of 98.3% and outperform a set of state-of-the-art and baseline models. The developed models are further evaluated with a real-world brain typing BCI and achieve a recognition accuracy of 93% over five instruction intentions suggesting good generalization over different kinds of intentions and BCI systems.

Making Sense of Spatio-Temporal Preserving Representations for EEG-Based Human Intention Recognition

Brain-machine interfaces (BMIs) can be used to decode brain activity into commands to control external devices. This paper presents the decoding of intuitive upper extremity imagery for multi-directional arm reaching tasks in three-dimensional (3D) environments. We designed and implemented an experimental environment in which electroencephalogram (EEG) signals can be acquired for movement execution and imagery. Fifteen subjects participated in our experiments. We proposed a multi-directional convolution neural network-bidirectional long short-term memory network (MDCBN)-based deep learning framework. The decoding performances for six directions in 3D space were measured by the correlation coefficient (CC) and the normalized root mean square error (NRMSE) between predicted and baseline velocity profiles. The grand-averaged CCs of multi-direction were 0.47 and 0.45 for the execution and imagery sessions, respectively, across all subjects. The NRMSE values were below 0.2 for both sessions. Furthermore, in this study, the proposed MDCBN was evaluated by two online experiments for real-time robotic arm control, and the grand-averaged success rates were approximately 0.60 (±0.14) and 0.43 (±0.09), respectively. Hence, we demonstrate the feasibility of intuitive robotic arm control based on EEG signals for real-world environments.

Brain-Controlled Robotic Arm System Based on Multi-Directional CNN-BiLSTM Network Using EEG Signals

Deep learning for motor imagery EEG-based classification: A review

Classifying limb movements using brain activity is an important task in Brain-computer Interfaces (BCI) that has been successfully used in multiple application domains, ranging from human-computer interaction to medical and biomedical applications. This paper proposes a novel solution for classification of left/right hand movement by exploiting a Long Short-Term Memory (LSTM) network with attention mechanism to learn the electroencephalogram (EEG) time-series information. To this end, a wide range of time and frequency domain features are extracted from the EEG signals and used to train an LSTM network to perform the classification task. We conduct extensive experiments with the EEG Movement dataset and show that our proposed solution our method achieves improvements over several benchmarks and state-of-the-art methods in both intra-subject and cross-subject validation schemes. Moreover, we utilize the proposed framework to analyze the information as received by the sensors and monitor the activated regions of the brain by tracking EEG topography throughout the experiments.

Classification of Hand Movements From EEG Using a Deep Attention-Based LSTM Network

Classification of motor imagery electroencephalograph signals is a fundamental problem in brain–computer interface (BCI) systems. We propose in this paper a classification framework based on long short-term memory (LSTM) networks. To achieve robust classification, a one dimension-aggregate approximation (1d-AX) is employed to extract effective signal representation for LSTM networks. Inspired by classical common spatial pattern, channel weighting technique is further deployed to enhance the effectiveness of the proposed classification framework. Public BCI competition data are used for the evaluation of the proposed feature extraction and classification network, whose performance is also compared with that of the state-of-the-arts approaches based on other deep networks.

LSTM-Based EEG Classification in Motor Imagery Tasks

The invention discloses a speech data processing method based on a variational Gaussian regression process Speech data processing is performed based on a variational Gaussian regression process model According to the variational Gaussian regression process model, based on a VFE approximation, transformation is performed on a log likelihood function, so that a solved final lower limit becomes large, and free variation Gaussian distribution of an active point set is obtained, the free variation Gaussian distribution is posterior distribution of selected points in the mapping approximation, andvoice data is processed in a mapping approximation method The speech data processing method improves the efficiency of calculation, subitems in the lower limit maximum values obtained lastly are approximated, and the calculation efficiency can be greatly improved while the loss of similarity is reduced to the largest extent

Speech data processing method based on variational Gaussian regression process

The invention discloses a block Gaussian regression model subset modeling method for voice data based on clustering. Voice data is framed according to the time interval, and the features of each frameof data are extracted. After the high-dimensional data extracted from each frame is reduced to three dimensions, a block Gaussian regression model is built. The block Gaussian regression model is a Gaussian regression process model based on a subset method. In a training data set, a plurality of points representing the original training data set are selected according to a greedy algorithm to constitute a subset, and data points carrying redundant information are removed. The selected subset is adjusted according to a specific test point. After clustering, the selected subset is from different clusters and can represent the whole data set. The subset selected in this way is composed of part of local optimal points nearest to the test point, and overall consideration is achieved.

Wang Ping

Papers

LSTM-Based EEG Classification in Motor Imagery Tasks

Speech data processing method based on variational Gaussian regression process

Block Gaussian regression model subset modeling method for voice data based on clustering