Emotion recognition based on electroencephalography (EEG) is a significant task in the brain-computer interface field. Recently, many deep learning-based emotion recognition methods are demonstrated to outperform traditional methods. However, it remains challenging to extract discriminative features for EEG emotion recognition, and most methods ignore useful information in channel and time. This paper proposes an attention-based convolutional recurrent neural network (ACRNN) to extract more discriminative features from EEG signals and improve the accuracy of emotion recognition. First, the proposed ACRNN adopts a channel-wise attention mechanism to adaptively assign the weights of different channels, and a CNN is employed to extract the spatial information of encoded EEG signals. Then, to explore the temporal information of EEG signals, extended self-attention is integrated into an RNN to recode the importance based on intrinsic similarity in EEG signals. We conducted extensive experiments on the DEAP and DREAMER databases. The experimental results demonstrate that the proposed ACRNN outperforms state-of-the-art methods.

EEG-based Emotion Recognition via Channel-wise Attention and Self Attention

Brain-Computer interfaces (BCIs) enhance the capability of human brain activities to interact with the environment. Recent advancements in technology and machine learning algorithms have increased interest in electroencephalographic (EEG)-based BCI applications. EEG-based intelligent BCI systems can facilitate continuous monitoring of fluctuations in human cognitive states under monotonous tasks, which is both beneficial for people in need of healthcare support and general researchers in different domain areas. In this review, we survey the recent literature on EEG signal sensing technologies and computational intelligence approaches in BCI applications, compensating for the gaps in the systematic summary of the past five years. Specifically, we first review the current status of BCI and signal sensing technologies for collecting reliable EEG signals. Then, we demonstrate state-of-the-art computational intelligence techniques, including fuzzy models and transfer learning in machine learning and deep learning algorithms, to detect, monitor, and maintain human cognitive states and task performance in prevalent applications. Finally, we present a couple of innovative BCI-inspired healthcare applications and discuss future research directions in EEG-based BCI research.

/pdf/eeg-based-brain-computer-interfaces-bcis-a-survey-of-recent-429m85k6zs.pdf

EEG-Based Brain-Computer Interfaces (BCIs): A Survey of Recent Studies on Signal Sensing Technologies and Computational Intelligence Approaches and Their Applications

Emotion recognition based on electroencephalography (EEG) is of great important in the field of Human–Computer Interaction (HCI), which has received extensive attention in recent years. Most traditional methods focus on extracting features in time domain and frequency domain. The spatial information from adjacent channels and symmetric channels is often ignored. To better learn spatial representation, in this paper, we propose an end-to-end Regional-Asymmetric Convolutional Neural Network (RACNN) for emotion recognition, which consists of temporal, regional and asymmetric feature extractors. Specifically, continuous 1D convolution layers are employed in temporal feature extractor to learn time–frequency representations. Then, regional feature extractor consists of two 2D convolution layers to capture regional information among physically adjacent channels. Meanwhile, we propose an Asymmetric Differential Layer (ADL) in asymmetric feature extractor by taking the asymmetry property of emotion responses into account, which can capture the discriminative information between left and right hemispheres of the brain. To evaluate our model, we conduct extensive experiments on two publicly available datasets, i.e., DEAP and DREAMER. The proposed model can obtain recognition accuracies over 95% for valence and arousal classification tasks on both datasets, significantly outperforming the state-of-the-art methods.

EEG-based emotion recognition using an end-to-end regional-asymmetric convolutional neural network

Recently, deep neural networks (DNNs) have been applied to emotion recognition tasks based on electroencephalography (EEG), and have achieved better performance than traditional algorithms. However, DNNs still have the disadvantages of too many hyperparameters and lots of training data. To overcome these shortcomings, in this article, we propose a method for multi-channel EEG-based emotion recognition using deep forest. First, we consider the effect of baseline signal to preprocess the raw artifact-eliminated EEG signal with baseline removal. Secondly, we construct 2 $D$ frame sequences by taking the spatial position relationship across channels into account. Finally, 2 $D$ frame sequences are input into the classification model constructed by deep forest that can mine the spatial and temporal information of EEG signals to classify EEG emotions. The proposed method can eliminate the need for feature extraction in traditional methods and the classification model is insensitive to hyperparameter settings, which greatly reduce the complexity of emotion recognition. To verify the feasibility of the proposed model, experiments were conducted on two public DEAP and DREAMER databases. On the DEAP database, the average accuracies reach to 97.69% and 97.53% for valence and arousal, respectively; on the DREAMER database, the average accuracies reach to 89.03%, 90.41%, and 89.89% for valence, arousal and dominance, respectively. These results show that the proposed method exhibits higher accuracy than the state-of-art methods.

Emotion Recognition From Multi-Channel EEG via Deep Forest

The seminal work on Affective Computing in 1995 by Picard set the base for computing that relates to, arises from, or influences emotions. Affective computing is a multidisciplinary field of research spanning the areas of computer science, psychology, and cognitive science. Potential applications include automated driver assistance, healthcare, human-computer interaction, entertainment, marketing, teaching and many others. Thus, quickly, the field acquired high interest, with an enormous growth of the number of papers published on the topic since its inception. This paper aims to (1) Present an introduction to the field of affective computing though the description of key theoretical concepts; (2) Describe the current state-of-the-art of emotion recognition, tracing the developments that helped foster the growth of the field; and lastly, (3) point the literature take-home messages and conclusions, evidencing the main challenges and future opportunities that lie ahead, in particular for the development of novel machine learning (ML) algorithms in the context of emotion recognition using physiological signals.

/pdf/a-review-current-challenges-and-future-possibilities-on-365dzvqva1.pdf

A Review, Current Challenges, and Future Possibilities on Emotion Recognition Using Machine Learning and Physiological Signals

As a challenging pattern recognition task, automatic real-time emotion recognition based on multi-channel EEG signals is becoming an important computer-aided method for emotion disorder diagnose in neurology and psychiatry. Traditional machine learning approaches require to design and extract various features from single or multiple channels based on comprehensive domain knowledge. Consequently, these approaches may be an obstacle for non-domain experts. On the contrast, deep learning approaches have been used successfully in many recent literatures to learn features and classify different types of data. In this paper, baseline signals are considered and a simple but effective pre-processing method has been proposed to improve the recognition accuracy. Meanwhile, a hybrid neural network which combines `Convolutional Neural Network (CNN)’ and `Recurrent Neural Network (RNN)’ has been applied to classify human emotion states by effectively learning compositional spatial-temporal representation of raw EEG streams. The CNN module is used to mine the inter-channel correlation among physically adjacent EEG signals by converting the chain-like EEG sequence into 2D-like frame sequence. The LSTM module is adopted to mine contextual information. Experiments are carried out in a segment-level emotion identification task, on the DEAP benchmarking dataset. Our experimental results indicate that the proposed pre-processing method can increase emotion recognition accuracy by 32% approximately and the model achieves a high performance with a mean accuracy of 90.80% and 91.03% on valence and arousal classification task respectively.

Emotion Recognition from Multi-Channel EEG through Parallel Convolutional Recurrent Neural Network

Automatic emotion recognition based on EEG is an important issue in Brain-Computer Interface (BCI) applications. In this paper, baseline signals were taken into account to improve recognition accuracy. Multi-Layer Perceptron (MLP), Decision Tree (DT) and our proposed approach were adopted to verify the effectiveness of baseline signals on classification results. Besides, a 3D representation of EEG segment was proposed to combine features of signals from different frequency bands while preserving spatial information among channels. The continuous convolutional neural network takes the constructed 3D EEG cube as input and makes prediction. Extensive experiments on public DEAP dataset indicate that the proposed method is well suited for emotion recognition tasks after considering the baseline signals. Our comparative experiments also confirmed that higher frequency bands of EEG signals can better characterize emotional states, and that the combination of features of multiple bands can complement each other and further improve the recognition accuracy.

Xiaowei Chen

Papers

Emotion Recognition from Multi-Channel EEG through Parallel Convolutional Recurrent Neural Network

Continuous Convolutional Neural Network with 3D Input for EEG-Based Emotion Recognition