scispace - formally typeset
Search or ask a question
Journal Article•DOI•

Investigating Critical Frequency Bands and Channels for EEG-Based Emotion Recognition with Deep Neural Networks

Wei-Long Zheng1, Bao-Liang Lu1•
Shanghai Jiao Tong University1
08 May 2015-IEEE Transactions on Autonomous Mental Development (IEEE)-Vol. 7, Iss: 3, pp 162-175
TL;DR: The experiment results show that neural signatures associated with different emotions do exist and they share commonality across sessions and individuals, and the performance of deep models with shallow models is compared.
Abstract: To investigate critical frequency bands and channels, this paper introduces deep belief networks (DBNs) to constructing EEG-based emotion recognition models for three emotions: positive, neutral and negative. We develop an EEG dataset acquired from 15 subjects. Each subject performs the experiments twice at the interval of a few days. DBNs are trained with differential entropy features extracted from multichannel EEG data. We examine the weights of the trained DBNs and investigate the critical frequency bands and channels. Four different profiles of 4, 6, 9, and 12 channels are selected. The recognition accuracies of these four profiles are relatively stable with the best accuracy of 86.65%, which is even better than that of the original 62 channels. The critical frequency bands and channels determined by using the weights of trained DBNs are consistent with the existing observations. In addition, our experiment results show that neural signatures associated with different emotions do exist and they share commonality across sessions and individuals. We compare the performance of deep models with shallow models. The average accuracies of DBN, SVM, LR, and KNN are 86.08%, 83.99%, 82.70%, and 72.60%, respectively.
Citations
More filters
Journal Article•DOI•
EEG-based Emotion Recognition via Channel-wise Attention and Self Attention

[...]

Wei Tao1, Chang Li1, Rencheng Song1, Juan Cheng1, Yu Liu1, Feng Wan2, Xun Chen1 •
Hefei University of Technology1, University of Macau2
22 Sep 2020-IEEE Transactions on Affective Computing
TL;DR: In this paper, an attention-based convolutional recurrent neural network (ACRNN) was proposed to extract more discriminative features from EEG signals and improve the accuracy of emotion recognition.
Abstract: Emotion recognition based on electroencephalography (EEG) is a significant task in the brain-computer interface field. Recently, many deep learning-based emotion recognition methods are demonstrated to outperform traditional methods. However, it remains challenging to extract discriminative features for EEG emotion recognition, and most methods ignore useful information in channel and time. This paper proposes an attention-based convolutional recurrent neural network (ACRNN) to extract more discriminative features from EEG signals and improve the accuracy of emotion recognition. First, the proposed ACRNN adopts a channel-wise attention mechanism to adaptively assign the weights of different channels, and a CNN is employed to extract the spatial information of encoded EEG signals. Then, to explore the temporal information of EEG signals, extended self-attention is integrated into an RNN to recode the importance based on intrinsic similarity in EEG signals. We conducted extensive experiments on the DEAP and DREAMER databases. The experimental results demonstrate that the proposed ACRNN outperforms state-of-the-art methods.

166 citations

Posted Content•
EEG-Based Emotion Recognition Using Regularized Graph Neural Networks

[...]

Peixiang Zhong1, Di Wang1, Chunyan Miao1•
Nanyang Technological University1
18 Jul 2019-arXiv: Computer Vision and Pattern Recognition
TL;DR: A regularized graph neural network for EEG-based emotion recognition that considers the biological topology among different brain regions to capture both local and global relations among different EEG channels and ablation studies show that the proposed adjacency matrix and two regularizers contribute consistent and significant gain to the performance of the model.
Abstract: Electroencephalography (EEG) measures the neuronal activities in different brain regions via electrodes Many existing studies on EEG-based emotion recognition do not fully exploit the topology of EEG channels In this paper, we propose a regularized graph neural network (RGNN) for EEG-based emotion recognition RGNN considers the biological topology among different brain regions to capture both local and global relations among different EEG channels Specifically, we model the inter-channel relations in EEG signals via an adjacency matrix in a graph neural network where the connection and sparseness of the adjacency matrix are inspired by neuroscience theories of human brain organization In addition, we propose two regularizers, namely node-wise domain adversarial training (NodeDAT) and emotion-aware distribution learning (EmotionDL), to better handle cross-subject EEG variations and noisy labels, respectively Extensive experiments on two public datasets, SEED and SEED-IV, demonstrate the superior performance of our model than state-of-the-art models in most experimental settings Moreover, ablation studies show that the proposed adjacency matrix and two regularizers contribute consistent and significant gain to the performance of our RGNN model Finally, investigations on the neuronal activities reveal important brain regions and inter-channel relations for EEG-based emotion recognition

158 citations

Cites background or methods or result from "Investigating Critical Frequency Ba..."

  • ...The SEED dataset [7] comprises EEG data of 15 subjects (7 males) recorded in 62 channels using the ESI NeuroScan System1....

    [...]

  • ...the asymmetry features of PSD [7] and functional connectivity [32], [33], where common indices such as correlation, coherence and phase synchronization were used estimate brain functional connectivity between channels....

    [...]

  • ...For SEED, we follow the experimental settings in [7], [12], [17] to evaluate our RGNN model using subject-dependent classification, i....

    [...]

  • ...This observation is consistent with the literature [7], [75]....

    [...]

  • ...3) We conduct extensive experiment in both subjectdependent and subject-independent classification settings on two public EEG datasets, namely SEED [7] and SEED-IV [25]....

    [...]

Journal Article•DOI•
Data Augmentation for Deep-Learning-Based Electroencephalography.

[...]

Elnaz Lashgari1, Dehua Liang1, Uri Maoz2•
Chapman University1, University of California, Los Angeles2
31 Jul 2020-Journal of Neuroscience Methods
TL;DR: DA increasingly used and considerably improved DL decoding accuracy on EEG and holds transformative promise for EEG processing, possibly like DL revolutionized computer vision, etc.

156 citations

Book Chapter•DOI•
Data Augmentation for EEG-Based Emotion Recognition with Deep Convolutional Neural Networks

[...]

Fang Wang1, Sheng-hua Zhong1, Jianfeng Peng1, Jianmin Jiang1, Yan Liu2 •
Shenzhen University1, Hong Kong Polytechnic University2
05 Feb 2018
TL;DR: The experimental results show that the simple data augmentation method can improve the performance of emotion recognition based on deep models effectively and is proposed to address the issue of data shortage in EEG-based emotion recognition.
Abstract: Emotion recognition is the task of recognizing a person’s emotional state. EEG, as a physiological signal, can provide more detailed and complex information for emotion recognition task. Meanwhile, EEG can’t be changed and hidden intentionally makes EEG-based emotion recognition achieve more effective and reliable result. Unfortunately, due to the cost of data collection, most EEG datasets have small number of EEG data. The lack of data makes it difficult to predict the emotion states with the deep models, which requires enough number of training data. In this paper, we propose to use a simple data augmentation method to address the issue of data shortage in EEG-based emotion recognition. In experiments, we explore the performance of emotion recognition with the shallow and deep computational models before and after data augmentation on two standard EEG-based emotion datasets. Our experimental results show that the simple data augmentation method can improve the performance of emotion recognition based on deep models effectively.

155 citations

Cites background or methods or result from "Investigating Critical Frequency Ba..."

  • ...It has been shown that DE features can obtain the superior performance in comparison with other commonly used features [1, 22]....

    [...]

  • ...It has been proven that alpha, beta and gamma bands of EEG are more predictive to the emotional states compared with the delta and theta bands [1, 17]....

    [...]

  • ...As an interdisciplinary field, the research of emotional recognition is benefited from the development of psychology, modern neuroscience, cognitive science, and computer science as well [1]....

    [...]

  • ...Models SVM [1] PCA + SVM (95% energy) PCA [22] + SVM (160 dimensions) PCA [22] + SVM (210 dimensions) LeNet ResNet...

    [...]

  • ...As a commonly used machine learning model, the basic idea of SVM is to map the input data to a high-dimensional feature space via a kernel transfer function, in this new space, these input data will be easier to be separated than that in the original feature space [1]....

    [...]

Journal Article•DOI•
A Bi-Hemisphere Domain Adversarial Neural Network Model for EEG Emotion Recognition

[...]

Yang Li1, Wenming Zheng1, Yuan Zong1, Zhen Cui2, Tong Zhang1, Xiaoyan Zhou3 •
Southeast University1, Nanjing University of Science and Technology2, Nanjing University3
01 Apr 2021-IEEE Transactions on Affective Computing
TL;DR: A novel neural network model, called bi-hemisphere domain adversarial neural network (BiDANN), inspired by the neuroscience findings that the left and right hemispheres of human's brain are asymmetric to the emotional response, and an improved version, denoted by BiDANN-S, for subject-independent EEG emotion recognition problem.
Abstract: In this paper, we propose a novel neural network model, called bi-hemisphere domain adversarial neural network (BiDANN) model, for electroencephalograph (EEG) emotion recognition. The BiDANN model is inspired by the neuroscience findings that the left and right hemispheres of human's brain are asymmetric to the emotional response. It contains a global and two local domain discriminators that work adversarially with a classifier to learn discriminative emotional features for each hemisphere. At the same time, it tries to reduce the possible domain differences in each hemisphere between the source and target domains so as to improve the generality of the recognition model. In addition, we also propose an improved version of BiDANN, denoted by BiDANN-S, for subject-independent EEG emotion recognition problem by lowering the influences of the personal information of subjects to the EEG emotion recognition. Extensive experiments on the SEED database are conducted to evaluate the performance of both BiDANN and BiDANN-S. The experimental results have shown that the proposed BiDANN and BiDANN models achieve state-of-the-art performance in the EEG emotion recognition.

147 citations

Cites methods from "Investigating Critical Frequency Ba..."

  • ...The SEED database contains 15 subjects’ EEG signals recorded from 62 electrode channels using ESI NeuroScan with a sampling rate of 1000 Hz....

    [...]

  • ...The confusion matrices of the subject-independent EEG emotion recognition results using BiDANN and BiDANN-S method on the SEED database....

    [...]

  • ...Extensive experiments on the SEED database are conducted to evaluate the performance of both BiDANN and BiDANNS....

    [...]

  • ...[13] by using 9 trails of EEG data per session of each subject as source (training) domain data whereas using the remaining 6 trials per session as target (testing) domain data....

    [...]

  • ...TABLE 5 The Mean Accuracies (and Standard Deviations) Using Different Frequency Bands for Subject-Independent EEG Emotion Recognition on SEED Database Methods Frequency bands d u a b g KLIEP [41] 39.22 (11.31) 35.98 (07.50) 33.31 (06.60) 44.47 (12.89) 42.05 (12.65) ULSIF [42] 41.32 (11.30) 36.27 (06.84) 38.94 (08.30) 41.87 (13.64) 41.02 (11.65) STM [43] 44.16 (09.60) 40.89 (08.22) 40.37 (09.82) 42.09 (13.34) 47.97 (12.43) SVM [37] 43.06 (08.27) 40.07 (06.50) 43.97 (10.89) 48.63 (10.29) 51.59 (11.83) TCA [45] 44.10 (08.22) 41.26 (09.21) 42.93 (14.33) 43.93 (10.06) 48.43 (09.73) TKL [46] 48.36 (10.31) 52.60 (11.84) 52.89 (11.07) 55.47 (09.80) 59.81 (12.41) SA [47] 53.23 (07.47) 50.60 (08.31) 55.06 (10.60) 56.72 (10.78) 64.47 (14.96) GFK [48] 52.73 (11.90) 54.07 (06.78) 54.98 (11.49) 59.29 (10.75) 66.92 (10.97) DGCNN [14] 49.79 (10.94) 46.36 (12.06) 48.29 (12.28) 56.15 (14.01) 54.87 (17.53) DANN [17] 56.66 (06.48) 54.95 (10.45) 59.37 (10.57) 67.14 (07.10) 71.30 (10.84) BiDANN-R1 57.33 (06.76) 57.00 (08.92) 58.20 (13.50) 64.76 (13.79) 65.15 (14.14) BiDANN-R2 59.67 (10.48) 60.70 (07.42) 61.08 (10.77) 74.09 (11.54) 72.77 (11.51) BiDANN 62.04 (06.64) 62.13 (07.37) 63.31 (11.46) 73.55 (08.83) 73.25 (09.21) BiDANN-S 63.01 (07.49) 63.22 (07.52) 63.50 (09.50) 73.59 (09.12) 73.72 (08.67) Denotes the experiment results obtained are based on our own implementation....

    [...]

References
More filters
Proceedings Article•
ImageNet Classification with Deep Convolutional Neural Networks

[...]

Alex Krizhevsky1, Ilya Sutskever1, Geoffrey E. Hinton1•
University of Toronto1
03 Dec 2012
TL;DR: The state-of-the-art performance of CNNs was achieved by Deep Convolutional Neural Networks (DCNNs) as discussed by the authors, which consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax.
Abstract: We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0% which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overriding in the fully-connected layers we employed a recently-developed regularization method called "dropout" that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry.

73,978 citations

Journal Article•DOI•
LIBSVM: A library for support vector machines

[...]

Chih-Chung Chang1, Chih-Jen Lin1•
National Taiwan University1
06 May 2011-ACM Transactions on Intelligent Systems and Technology
TL;DR: Issues such as solving SVM optimization problems theoretical convergence multiclass classification probability estimates and parameter selection are discussed in detail.
Abstract: LIBSVM is a library for Support Vector Machines (SVMs). We have been actively developing this package since the year 2000. The goal is to help users to easily apply SVM to their applications. LIBSVM has gained wide popularity in machine learning and many other areas. In this article, we present all implementation details of LIBSVM. Issues such as solving SVM optimization problems theoretical convergence multiclass classification probability estimates and parameter selection are discussed in detail.

40,826 citations

"Investigating Critical Frequency Ba..." refers methods in this paper

  • ...We use LIBSVM software [56] to implement the SVM classifier and employ linear kernel....

    [...]

Journal Article•DOI•
Reducing the Dimensionality of Data with Neural Networks

[...]

Geoffrey E. Hinton1, Ruslan Salakhutdinov1•
University of Toronto1
28 Jul 2006-Science
TL;DR: In this article, an effective way of initializing the weights that allows deep autoencoder networks to learn low-dimensional codes that work much better than principal components analysis as a tool to reduce the dimensionality of data is described.
Abstract: High-dimensional data can be converted to low-dimensional codes by training a multilayer neural network with a small central layer to reconstruct high-dimensional input vectors. Gradient descent can be used for fine-tuning the weights in such "autoencoder" networks, but this works well only if the initial weights are close to a good solution. We describe an effective way of initializing the weights that allows deep autoencoder networks to learn low-dimensional codes that work much better than principal components analysis as a tool to reduce the dimensionality of data.

16,717 citations

Journal Article•DOI•
A fast learning algorithm for deep belief nets

[...]

Geoffrey E. Hinton1, Simon Osindero1, Yee Whye Teh2•
University of Toronto1, National University of Singapore2
01 Jul 2006-Neural Computation
TL;DR: A fast, greedy algorithm is derived that can learn deep, directed belief networks one layer at a time, provided the top two layers form an undirected associative memory.
Abstract: We show how to use "complementary priors" to eliminate the explaining-away effects that make inference difficult in densely connected belief nets that have many hidden layers. Using complementary priors, we derive a fast, greedy algorithm that can learn deep, directed belief networks one layer at a time, provided the top two layers form an undirected associative memory. The fast, greedy algorithm is used to initialize a slower learning procedure that fine-tunes the weights using a contrastive version of the wake-sleep algorithm. After fine-tuning, a network with three hidden layers forms a very good generative model of the joint distribution of handwritten digit images and their labels. This generative model gives better digit classification than the best discriminative learning algorithms. The low-dimensional manifolds on which the digits lie are modeled by long ravines in the free-energy landscape of the top-level associative memory, and it is easy to explore these ravines by using the directed connections to display what the associative memory has in mind.

15,055 citations

"Investigating Critical Frequency Ba..." refers background in this paper

  • ...MLP, SVMs, CRFs) in many challenge tasks, especially in speech and image domains [29]–[31]....

    [...]

  • ...Many deep architecture models are proposed such as deep auto-encoder [26], convolution neural network [27], [28] and deep belief network [29]....

    [...]

  • ...Deep Belief Network is a probabilistic generative model with deep architecture, which characterizes the input data distribution using hidden variables [25], [29]....

    [...]

Book•
Neural Networks And Learning Machines

[...]

Simon Haykin
01 Jan 2010
TL;DR: Refocused, revised and renamed to reflect the duality of neural networks and learning machines, this edition recognizes that the subject matter is richer when these topics are studied together.
Abstract: For graduate-level neural network courses offered in the departments of Computer Engineering, Electrical Engineering, and Computer Science. Neural Networks and Learning Machines, Third Edition is renowned for its thoroughness and readability. This well-organized and completely upto-date text remains the most comprehensive treatment of neural networks from an engineering perspective. This is ideal for professional engineers and research scientists. Matlab codes used for the computer experiments in the text are available for download at: http://www.pearsonhighered.com/haykin/ Refocused, revised and renamed to reflect the duality of neural networks and learning machines, this edition recognizes that the subject matter is richer when these topics are studied together. Ideas drawn from neural networks and machine learning are hybridized to perform improved learning tasks beyond the capability of either independently.

4,943 citations

"Investigating Critical Frequency Ba..." refers background in this paper

  • ...According to the rules of knowledge representation, if a particular feature is important, there should be a larger number of neurons involved in representing it in the network [59]....

    [...]

Related Papers (5)
DEAP: A Database for Emotion Analysis ;Using Physiological Signals

[...]

01 Jan 2012-IEEE Transactions on Affective Computing
Sander Koelstra, Christian Mühl, Mohammad Soleymani, Jong-Seok Lee, Ashkan Yazdani, Touradj Ebrahimi, Thierry Pun, Anton Nijholt, Ioannis Patras 
Differential entropy feature for EEG-based emotion classification

[...]

01 Nov 2013
Ruo-Nan Duan, Jia-Yi Zhu, Bao-Liang Lu
Feature Extraction and Selection for Emotion Recognition from EEG

[...]

01 Jul 2014-IEEE Transactions on Affective Computing
Robert Jenke, Angelika Peer, Martin Buss
EEG-Based Emotion Recognition in Music Listening

[...]

03 May 2010-IEEE Transactions on Biomedical Engineering
Yuan-Pin Lin, Chi-Hong Wang, Tzyy-Ping Jung, Tien-Lin Wu, Shyh-Kang Jeng, Jeng Ren Duann, Jyh-Horng Chen 
Emotions Recognition Using EEG Signals: A Survey

[...]

01 Jul 2019-IEEE Transactions on Affective Computing
Soraia M. Alarcão, Manuel J. Fonseca