Objective: Most current Electroencephalography (EEG)-based Brain-Computer Interfaces (BCIs) are based on machine learning algorithms. There is a large diversity of classifier types that are used in this field, as described in our 2007 review paper. Now, approximately 10 years after this review publication, many new algorithms have been developed and tested to classify EEG signals in BCIs. The time is therefore ripe for an updated review of EEG classification algorithms for BCIs. Approach: We surveyed the BCI and machine learning literature from 2007 to 2017 to identify the new classification approaches that have been investigated to design BCIs. We synthesize these studies in order to present such algorithms, to report how they were used for BCIs, what were the outcomes, and to identify their pros and cons. Main results: We found that the recently designed classification algorithms for EEG-based BCIs can be divided into four main categories: adaptive classifiers, matrix and tensor classifiers, transfer learning and deep learning, plus a few other miscellaneous classifiers. Among these, adaptive classifiers were demonstrated to be generally superior to static ones, even with unsupervised adaptation. Transfer learning can also prove useful although the benefits of transfer learning remain unpredictable. Riemannian geometry-based methods have reached state-of-the-art performances on multiple BCI problems and deserve to be explored more thoroughly, along with tensor-based methods. Shrinkage linear discriminant analysis and random forests also appear particularly useful for small training samples settings. On the other hand, deep learning methods have not yet shown convincing improvement over state-of-the-art BCI methods. Significance: This paper provides a comprehensive overview of the modern classification algorithms used in EEG-based BCIs, presents the principles of these Review of Classification Algorithms for EEG-based BCI 2 methods and guidelines on when and how to use them. It also identifies a number of challenges to further advance EEG classification in BCI.

https://iopscience.iop.org/article/10.1088/1741-2552/aab2f2/pdf

A Review of Classification Algorithms for EEG-based Brain-Computer Interfaces: A 10-year Update

Despite significant successes achieved in knowledge discovery, traditional machine learning methods may fail to obtain satisfactory performances when dealing with complex data, such as imbalanced, high-dimensional, noisy data, etc. The reason behind is that it is difficult for these methods to capture multiple characteristics and underlying structure of data. In this context, it becomes an important topic in the data mining field that how to effectively construct an efficient knowledge discovery and mining model. Ensemble learning, as one research hot spot, aims to integrate data fusion, data modeling, and data mining into a unified framework. Specifically, ensemble learning firstly extracts a set of features with a variety of transformations. Based on these learned features, multiple learning algorithms are utilized to produce weak predictive results. Finally, ensemble learning fuses the informative knowledge from the above results obtained to achieve knowledge discovery and better predictive performance via voting schemes in an adaptive way. In this paper, we review the research progress of the mainstream approaches of ensemble learning and classify them based on different characteristics. In addition, we present challenges and possible research directions for each mainstream approach of ensemble learning, and we also give an extra introduction for the combination of ensemble learning with other machine learning hot spots such as deep learning, reinforcement learning, etc.

A survey on ensemble learning

Emotion recognition using multi-modal data and machine learning techniques: A tutorial and review

The effects of adaptive task allocation on monitoring for automation failure during multitask flight simulation were examined. Participants monitored an automated engine status task while simultaneously performing tracking and fuel management tasks over three 30-min sessions. Two methods of adaptive task allocation, both involving temporary return of the automated engine status task to the human operator ("human control"), were examined as a possible countermeasure to monitoring inefficiency. For the model-based adaptive group, the engine status task was allocated to all participants in the middle of the second session for 10 min, following which it was again returned to automation control. The same occurred for the performance-based adaptive group, but only if an individual participant's monitoring performance up to that point did not meet a specified criterion. For the nonadaptive control groups, the engine status task remained automated throughout the experiment. All groups had low probabilities of detection of automation failures for the first 40 min spent with automation. However, following the 10-min intervening period of human control, both adaptive groups detected significantly more automation failures during the subsequent blocks under automation control. The results show that adaptive task allocation can enhance monitoring of automated systems. Both model-based and performance-based allocation improved monitoring of automation. Implications for the design of automated systems are discussed.

Effects of adaptive task allocation on monitoring of automated systems

Emotion recognition based on multi-channel electroencephalograph (EEG) signals is becoming increasingly attractive. However, the conventional methods ignore the spatial characteristics of EEG signals, which also contain salient information related to emotion states. In this paper, a deep learning framework based on a multiband feature matrix (MFM) and a capsule network (CapsNet) is proposed. In the framework, the frequency domain, spatial characteristics, and frequency band characteristics of the multi-channel EEG signals are combined to construct the MFM. Then, the CapsNet model is introduced to recognize emotion states according to the input MFM. Experiments conducted on the dataset for emotion analysis using EEG, physiological, and video signals (DEAP) indicate that the proposed method outperforms most of the common models. The experimental results demonstrate that the three characteristics contained in the MFM were complementary and the capsule network was more suitable for mining and utilizing the three correlation characteristics.

https://www.mdpi.com/1424-8220/19/9/2212/pdf

Emotion Recognition from Multiband EEG Signals Using CapsNet

Recognition of emotions using multimodal physiological signals and an ensemble deep learning model

Physiological-signal-based mental workload estimation via transfer dynamical autoencoders in a deep learning framework

Objective. Brain computer interface (BCI) technology is an innovative way of information exchange, which can effectively convert physiological signals into control instructions of machines. Due to its spontaneity and device independence, the motor imagery (MI) electroencephalography (EEG) signal is used as a common BCI signal source to achieve direct control of external devices. Several online MI EEG-based systems have shown potential for rehabilitation. However, the generalization ability of the current classification model of MI tasks is still limited and the real-time prototype is far from widespread in practice. Approach. To solve these problems, this paper proposes an optimized neural network architecture based on our previous work. Firstly, the artifact components in the MI-EEG signal are removed by using the threshold and threshold function related to the artifact removal evaluation index, and then the data is augmented by the empirical mode decomposition (EMD) algorithm. Furthermore, the ensemble learning (EL) method and fine-tuning strategy in transfer learning (TL) are used to optimize the classification model. Finally, combined with the flexible binary encoding strategy, the EEG signal recognition results are mapped to the control commands of the robotic arm, which realizes multiple degrees of freedom control of the robotic arm. Main results. The results show that EMD has an obvious data amount enhancement effect on a small dataset, and the EL and TL can improve intra-subject and inter-subject model evaluation performance, respectively. The use of a binary coding method realizes the expansion of control instructions, i.e. four kinds of MI-EEG signals are used to complete the control of 7 degrees of freedom of the robotic arm. Significance. Our work not only improves the classification accuracy of the subject and the generality of the classification model while also extending the BCI control instruction set.

Flexible coding scheme for robotic arm control driven by motor imagery decoding

Pronoun resolution plays an important role in language comprehension. However, little is known about its recruited cognitive mechanisms. Our investigation aims to explore the cognitive mechanisms underlying various types of pronoun resolution in Chinese using an electroencephalograph (EEG). We used three convolutional neural networks (CNNs)—LeNeT-5, GoogleNet, and EffifcientNet—to discover high-level feature abstractions of the EEG spatial topologies. The output of the three models was then fused using different scales by principal component analysis (PCA) to achieve cognitive workload classification. Overall, the workload classification rate by fusing three deep networks can be achieved at 55–63% in a participant-specific manner. We provide evidence that both the behavioral indicator of reaction time and the neural indicator of cognitive workload collected during pronoun resolution vary depending on the type of the pronoun. We observed an increase in reaction time accompanied by a decrease of the theta power while participants were processing ambiguous pronoun resolution compared to unambiguous controls. We propose that ambiguous pronoun resolution involves a more time-consuming yet more flexible cognitive mechanism, consistent with the predictions of the decision-making framework from an influential pragmatic tradition. Our results extend previous research that the cognitive states of resolving ambiguous and unambiguous pronouns are differentiated, indicating that cognitive workload evaluated using the method of machine learning for analysis of EEG signals acts as a complementary indicator for studying pronoun resolution and serves as an important inspiration for human–machine interaction.

/pdf/assessing-distinct-cognitive-workload-levels-associated-with-3hnk27c1.pdf

Mengyuan Zhao

Papers

Recognition of emotions using multimodal physiological signals and an ensemble deep learning model

Physiological-signal-based mental workload estimation via transfer dynamical autoencoders in a deep learning framework

Flexible coding scheme for robotic arm control driven by motor imagery decoding

Assessing Distinct Cognitive Workload Levels Associated with Unambiguous and Ambiguous Pronoun Resolutions in Human–Machine Interactions

Identification of human mental workload levels in a language comprehension task with imbalance neurophysiological data