A novel approach for analyzing human emotions based on electroencephalography (EEG)
01 Apr 2017-pp 1-6
TL;DR: This work considers EEG signals for emotion recognition which not only ignores external factors but also helps to detect real emotions arising directly from the authors' brain.
Abstract: In our day-to-day life Emotions play an vital role. They help in identifying a human state of mind. Information about the human state of mind can significantly help in human-machine interaction and brain-computer interface. Some of the existing researchers have used speech, text, gesture or facial expressions for emotion recognition. However, these factors vary across culture and nation. Because of which, it is difficult to detect emotions more accurately. Hence, present work considers EEG signals for emotion recognition which not only ignores external factors but also helps to detect real emotions arising directly from our brain. A benchmark DEAP (Dataset for Emotion Analysis using Physiological signals) dataset is used for emotion investigation. A novel feature extraction technique called Frequency Cepstral Coefficient (FCC) is proposed to extract features from DEAP dataset. FCC technique is compared with Kernel Density Estimation (KDE) on DEAP dataset for feature extraction. These extracted features are then classified into two emotional states — happy and sad using K-Nearest Neighbor (K-NN) classifier. The selection of most appropriate and reliable method of feature extraction greatly helps for accurate classification. Number of experiments was conducted to evaluate the efficiency of feature extraction techniques. The experimental results show that KDE gives 80% accuracy and FCC outperforms it by achieving 90% accuracy on DEAP.
Citations
More filters
TL;DR: Wang et al. as mentioned in this paper proposed a method based on convolutional neural network with data augmentation method Borderline-synthetic minority oversampling technique, which is proved to be effective in emotion recognition, and the average accuracy rate of 32 subjects on valence and arousal are 97.47% and 97.76% respectively.
Abstract: In recent years, with the continuous development of artificial intelligence and brain-computer interface technology, emotion recognition based on physiological signals, especially electroencephalogram signals, has become a popular research topic and attracted wide attention. However, the imbalance of the data sets themselves, affective features’ extraction from electroencephalogram signals, and the design of classifiers with excellent performance, pose a great challenge to the subject. Motivated by the outstanding performance of deep learning approaches in pattern recognition tasks, we propose a method based on convolutional neural network with data augmentation method Borderline-synthetic minority oversampling technique. First, we obtain 32-channel electroencephalogram signals from DEAP data set, which is the standard data set of emotion recognition. Then, after data pre-processing, we extract features in frequency domain and data augmentation based on the data augmentation algorithm above for getting more balanced data. Finally, we train a one dimensional convolutional neural network for three classification on two emotional dimensions valence and arousal. Meanwhile, the proposed method is compared with some traditional machine learning methods and some existing methods by other researchers, which is proved to be effective in emotion recognition, and the average accuracy rate of 32 subjects on valence and arousal are 97.47% and 97.76% respectively. Compared with other existing methods, the performance of the proposed method with data augmentation algorithm Borderline-SMOTE shows its advantage in affective emotional recognition than that without Borderline-SMOTE.
27 citations
TL;DR: A method to accurately recognize six emotions using ECG and EDA signals and applying autoregressive hidden Markov models (AR-HMMs) and heart rate variability analysis on these signals is presented.
18 citations
TL;DR: The proposed emotion recognition method of EEG signals based on the ensemble learning method, AdaBoost, is proved to be effective in emotion recognition, and the best average accuracy rate can reach up to 88.70% on the dominance dimension.
Abstract: In recent years, with the continuous development of artificial intelligence and brain-computer interface technology, emotion recognition based on physiological signals, especially, electroencephalogram (EEG) signals, has become a popular research topic and attracted wide attention. However, how to extract effective features from EEG signals and accurately recognize them by classifiers have also become an increasingly important task. Therefore, in this paper, we propose an emotion recognition method of EEG signals based on the ensemble learning method, AdaBoost. First, we consider the time domain, time-frequency domain, and nonlinear features related to emotion, extract them from the preprocessed EEG signals, and fuse the features into an eigenvector matrix. Then, the linear discriminant analysis feature selection method is used to reduce the dimensionality of the features. Next, we use the optimized feature sets and train a classifier based on the ensemble learning method, AdaBoost, for binary classification. Finally, the proposed method has been tested in the DEAP data set on four emotional dimensions: valence, arousal, dominance, and liking. The proposed method is proved to be effective in emotion recognition, and the best average accuracy rate can reach up to 88.70% on the dominance dimension. Compared with other existing methods, the performance of the proposed method is significantly improved.
13 citations
Cites background from "A novel approach for analyzing huma..."
...Meanwhile, with the continuous development of artificial intelligence and brain-computer interface technology [12, 13], emotion recognition based on physiological signals, especially on electroencephalogram (EEG) signals, has gradually become the mainstream of emotion recognition [14, 15]....
[...]
...-e idea of emotion recognition research based on EEG signals can be summarized as data preprocessing, feature extraction, classification, and evaluation of the model’s performance [15]....
[...]
TL;DR: It is proved that, single-channel EEG contains sufficient information for emotion recognition and, accuracy achieved using proposed method of single channel (FP1) is almost equivalent to the accuracy of 32 channels.
Abstract: Anxiety, nervousness and stress are daily challenges for mankind. These challenges are very severe specifically for students of the age group between years 14 to 25. Therefore it very important to develop the simplest, low cost, accurate and handy process which will be helpful for the society to gauge the anxiety levels and take necessary corrective actions to avoid health and psychological issues. It is of extreme importance to have regular checks on change in behavior and to ensure correct emotion analysis and take the corrective action. Article elaborates unique feature extraction method called as “linear formulation of differential entropy”. With this method we have significantly reduced number of (Electroencephalography) EEG channels for emotion detection. This work has discovered new approach in neuroscience. It’s proved that, single-channel EEG contains sufficient information for emotion recognition. The performance of the newly proposed technique is based on the “Database for Emotion Analysis using Physiological Signals” (DEAP) benchmark database using single channel FP1, prefrontal channel [FP1, AF3, FP2, AF4], and all 32 channel. Bidirectional long short term memory (BiLSTM) is used as classifier. The performance shows that, accuracy achieved using proposed method of single channel (FP1) is almost equivalent to the accuracy of 32 channels.
9 citations
20 Sep 2019
TL;DR: A comprehensive survey is made on feature extraction methods and their comparative merits and limitations of EEG based automatic emotion recognition.
Abstract: In recent times, emotion recognition is in attention in brain computer interface (BCI) and human computer interaction (HCI) research area to provide a very good communication between brain and computer. The aim is to achieve a good recognition rate, although there are numerous researches have been conducted also there has been created several confusions with the definition of human emotions and the difference between emotions and moods. To detect brain signal, Electroencephalogram (EEG) signal has become biological marker. For its low cost, good time and spatial resolution EEG has been used widely in BCI researches. Extraction of features from EEG signals is one of the vital steps of EEG based emotion recognition. The appropriate feature selection for EEG based automatic emotion recognition is still now a big research topic. In this paper, a comprehensive survey is made on feature extraction methods and their comparative merits and limitations.
6 citations
References
More filters
TL;DR: A multimodal data set for the analysis of human affective states was presented and a novel method for stimuli selection is proposed using retrieval by affective tags from the last.fm website, video highlight detection, and an online assessment tool.
Abstract: We present a multimodal data set for the analysis of human affective states. The electroencephalogram (EEG) and peripheral physiological signals of 32 participants were recorded as each watched 40 one-minute long excerpts of music videos. Participants rated each video in terms of the levels of arousal, valence, like/dislike, dominance, and familiarity. For 22 of the 32 participants, frontal face video was also recorded. A novel method for stimuli selection is proposed using retrieval by affective tags from the last.fm website, video highlight detection, and an online assessment tool. An extensive analysis of the participants' ratings during the experiment is presented. Correlates between the EEG signal frequencies and the participants' ratings are investigated. Methods and results are presented for single-trial classification of arousal, valence, and like/dislike ratings using the modalities of EEG, peripheral physiological signals, and multimedia content analysis. Finally, decision fusion of the classification results from different modalities is performed. The data set is made publicly available and we encourage other researchers to use it for testing their own affective state estimation methods.
3,013 citations
"A novel approach for analyzing huma..." refers methods in this paper
...Experiments were conducted on 32 volunteers to record EEG and physiological signals to collect their rating [7]....
[...]
...Out of the 32 channels, our proposed system uses only prefrontal (FP1) channel for EEG analysis as it gives a better result for emotion recognition as per the existing survey [7]....
[...]
Proceedings Article•
01 Jan 2000
TL;DR: The results show that the use of the Mel scale for modeling music is at least not harmful for this problem, although further experimentation is needed to verify that this is the optimal scale in the general case and whether this transform is valid for music spectra.
Abstract: We examine in some detail Mel Frequency Cepstral Coefficients (MFCCs) the dominant features used for speech recognition and investigate their applicability to modeling music. In particular, we examine two of the main assumptions of the process of forming MFCCs: the use of the Mel frequency scale to model the spectra; and the use of the Discrete Cosine Transform (DCT) to decorrelate the Mel-spectral vectors. We examine the first assumption in the context of speech/music discrimination. Our results show that the use of the Mel scale for modeling music is at least not harmful for this problem, although further experimentation is needed to verify that this is the optimal scale in the general case. We investigate the second assumption by examining the basis vectors of the theoretically optimal transform to decorrelate music and speech spectral vectors. Our results demonstrate that the use of the DCT to decorrelate vectors is appropriate for both speech and music spectra. MFCCs for Music Analysis Of all the human generated sounds which influence our lives, speech and music are arguably the most prolific. Speech has received much focused attention and decades of research in this community have led to usable systems and convergence of the features used for speech analysis. In the music community however, although the field of synthesis is very mature, a dominant paradigm has yet to emerge to solve other problems such as music classification or transcription. Consequently, many representations for music have been proposed (e.g. (Martin1998), (Scheirer1997), (Blum1999)). In this paper, we examine some of the assumptions of Mel Frequency Cepstral Coefficients (MFCCs) the dominant features used for speech recognition and examine whether these assumptions are valid for modeling music. MFCCs have been used by other authors to model music and audio sounds (e.g. (Blum1999)). These works however use cepstral features merely because they have been so successful for speech recognition without examining the assumptions made in great detail. MFCCs (e.g. see (Rabiner1993)) are short-term spectral features. They are calculated as follows (the steps and assumptions made are explained in more detail in the full paper): 1. Divide signal into frames. 2. For each frame, obtain the amplitude spectrum. 3. Take the logarithm. 4. Convert to Mel (a perceptually-based) spectrum. 5. Take the discrete cosine transform (DCT). We seek to determine whether this process is suitable for creating features to model music. We examine only steps 4 and 5 since, as explained in the full paper, the other steps are less controversial. Step 4 calculates the log amplitude spectrum on the so-called Mel scale. This transformation emphasizes lower frequencies which are perceptually more meaningful for speech. It is possible however that the Mel scale may not be optimal for music as there may be more information in say higher frequencies. Step 5 takes the DCT of the Mel spectra. For speech, this approximates principal components analysis (PCA) which decorrelates the components of the feature vectors. We investigate whether this transform is valid for music spectra. Mel vs Linear Spectral Modeling To investigate the effect of using the Mel scale, we examine the performance of a simple speech/music discriminator. We use around 3 hours of labeled data from a broadcast news show, divided into 2 hours of training data and 40 minutes of testing data. We convert the data to ‘Mel’ and ‘Linear’ cepstral features and train mixture of Gaussian classifiers for each class. We then classify each segment in the test data using these models. This process is described in more detail in the full paper. We find that for this speech/music classification problem, the results are (statistically) significantly better if Mel-based cepstral features rather than linear-based cepstral features are used. However, whether this is simply because the Mel scale models speech better or because it also models music better is not clear. At worst, we can conclude that using the Mel cepstrum to model music in this speech/music discrimination problem is not harmful. Further tests are needed to verify that the Mel cepstrum is appropriate for modeling music in the general case. Using the DCT to Approximate Principal Components Analysis We additionally investigate the effectiveness of using the DCT to decorrelate Mel spectral features. The mathematically correct way to decorrelate components is to use PCA (or equivalently the KL transform). This transform uses the eigenvalues of the covariance matrix of the data to be modeled as basis vectors. By investigating how closely these vectors approximate cosine functions we can get a feel for how well the DCT approximates PCA. By inspecting the eigenvectors for the Mel log spectra for around 3 hours of speech and 4 hours of music we see that the DCT is an appropriate transform for decorrelating music (and speech) log spectra. Future Work Future work should focus on a more thorough examination the parameters used to generate MFCC features such as the sampling rate of the signal, the frequency scaling (Mel or otherwise) and the number of bins to use when smoothing. Also worthy of investigation is the windowing size and frame rate. Suggested Readings Blum, T, Keislar, D., Wheaton, J. and Wold, E., 1999, Method and article of manufacture for content-based analysis, storage, retrieval, and segmentation of audio information, U.S. Patent 5, 918, 223. Martin, K.. 1998, Toward automatic sound source recognition: identifying musical instruments, Proceedings NATO Computational Hearing Advanced Study Institute. Rabiner, L. and Juang, B., 1993, Fundamentals of Speech Recognition, Prentice-Hall. Scheirer, E. and Slaney, M., 1997, Construction and evaluation of a robust multifeature speech/music discriminator, Proceedings IEEE International Conference on Acoustics, Speech, and Signal Processing.
1,189 citations
"A novel approach for analyzing huma..." refers background in this paper
...N is consider as the number of sampling points within a signal frame and the time frame [23]....
[...]
TL;DR: A novel scheme of emotion-specific multilevel dichotomous classification (EMDC) is developed and compared with direct multiclass classification using the pLDA, with improved recognition accuracy of 95 percent and 70 percent for subject-dependent and subject-independent classification, respectively.
Abstract: Little attention has been paid so far to physiological signals for emotion recognition compared to audiovisual emotion channels such as facial expression or speech. This paper investigates the potential of physiological signals as reliable channels for emotion recognition. All essential stages of an automatic recognition system are discussed, from the recording of a physiological data set to a feature-based multiclass classification. In order to collect a physiological data set from multiple subjects over many weeks, we used a musical induction method that spontaneously leads subjects to real emotional states, without any deliberate laboratory setting. Four-channel biosensors were used to measure electromyogram, electrocardiogram, skin conductivity, and respiration changes. A wide range of physiological features from various analysis domains, including time/frequency, entropy, geometric analysis, subband spectra, multiscale entropy, etc., is proposed in order to find the best emotion-relevant features and to correlate them with emotional states. The best features extracted are specified in detail and their effectiveness is proven by classification results. Classification of four musical emotions (positive/high arousal, negative/high arousal, negative/low arousal, and positive/low arousal) is performed by using an extended linear discriminant analysis (pLDA). Furthermore, by exploiting a dichotomic property of the 2D emotion model, we develop a novel scheme of emotion-specific multilevel dichotomous classification (EMDC) and compare its performance with direct multiclass classification using the pLDA. An improved recognition accuracy of 95 percent and 70 percent for subject-dependent and subject-independent classification, respectively, is achieved by using the EMDC scheme.
953 citations
TL;DR: This work presents the first such comprehensive survey of all BCI designs using electrical signal recordings published prior to January 2006, and asks what are the key signal processing components of a BCI, and what signal processing algorithms have been used in BCIs.
Abstract: Brain–computer interfaces (BCIs) aim at providing a non-muscular channel for sending commands to the external world using the electroencephalographic activity or other electrophysiological measures of the brain function. An essential factor in the successful operation of BCI systems is the methods used to process the brain signals. In the BCI literature, however, there is no comprehensive review of the signal processing techniques used. This work presents the first such comprehensive survey of all BCI designs using electrical signal recordings published prior to January 2006. Detailed results from this survey are presented and discussed. The following key research questions are addressed: (1) what are the key signal processing components of a BCI, (2) what signal processing algorithms have been used in BCIs and (3) which signal processing techniques have received more attention? S This article has associated online supplementary data files
844 citations
"A novel approach for analyzing huma..." refers background or methods in this paper
...‘Features Extraction Techniques of EEG Signal for BCI Applications’ Comp. and Inf....
[...]
...[10] Ali Bashashati, Mehrdad, Fatourechi, Rabab K Ward and Gary E Birch....
[...]
...BCI Conf., Graz, Austria, 2011....
[...]
...The designed interpretable BCI [10] using FIS Classifier show the same level of accuracy as compared to Linear Classifier, Multilayer Perception classifiers and Support Vector Machine....
[...]
...Affective computation on EEG emotion recognition system based on the Affective Brain-Computer Interface (ABCI)....
[...]
28 Jul 2003
TL;DR: The results of a linear (linear discriminant analysis) and two nonlinear classifiers applied to the classification of spontaneous EEG during five mental tasks are reported, showing that non linear classifiers produce only slightly better classification results.
Abstract: The reliable operation of brain-computer interfaces (BCIs) based on spontaneous electroencephalogram (EEG) signals requires accurate classification of multichannel EEG. The design of EEG representations and classifiers for BCI are open research questions whose difficulty stems from the need to extract complex spatial and temporal patterns from noisy multidimensional time series obtained from EEG measurements. The high-dimensional and noisy nature of EEG may limit the advantage of nonlinear classification methods over linear ones. This paper reports the results of a linear (linear discriminant analysis) and two nonlinear classifiers (neural networks and support vector machines) applied to the classification of spontaneous EEG during five mental tasks, showing that nonlinear classifiers produce only slightly better classification results. An approach to feature selection based on genetic algorithms is also presented with preliminary results of application to EEG during finger movement.
686 citations