Objective Electroencephalography (EEG) analysis has been an important tool in neuroscience with applications in neuroscience, neural engineering (e.g. Brain-computer interfaces, BCI's), and even commercial applications. Many of the analytical tools used in EEG studies have used machine learning to uncover relevant information for neural classification and neuroimaging. Recently, the availability of large EEG data sets and advances in machine learning have both led to the deployment of deep learning architectures, especially in the analysis of EEG signals and in understanding the information it may contain for brain functionality. The robust automatic classification of these signals is an important step towards making the use of EEG more practical in many applications and less reliant on trained professionals. Towards this goal, a systematic review of the literature on deep learning applications to EEG classification was performed to address the following critical questions: (1) Which EEG classification tasks have been explored with deep learning? (2) What input formulations have been used for training the deep networks? (3) Are there specific deep learning network structures suitable for specific types of tasks? Approach A systematic literature review of EEG classification using deep learning was performed on Web of Science and PubMed databases, resulting in 90 identified studies. Those studies were analyzed based on type of task, EEG preprocessing methods, input type, and deep learning architecture. Main results For EEG classification tasks, convolutional neural networks, recurrent neural networks, deep belief networks outperform stacked auto-encoders and multi-layer perceptron neural networks in classification accuracy. The tasks that used deep learning fell into five general groups: emotion recognition, motor imagery, mental workload, seizure detection, event related potential detection, and sleep scoring. For each type of task, we describe the specific input formulation, major characteristics, and end classifier recommendations found through this review. Significance This review summarizes the current practices and performance outcomes in the use of deep learning for EEG classification. Practical suggestions on the selection of many hyperparameters are provided in the hope that they will promote or guide the deployment of deep learning to EEG datasets in future research.

https://iopscience.iop.org/article/10.1088/1741-2552/ab0ab5/pdf

Deep learning for electroencephalogram (EEG) classification tasks: a review.

Context Electroencephalography (EEG) is a complex signal and can require several years of training, as well as advanced signal processing and feature extraction methodologies to be correctly interpreted. Recently, deep learning (DL) has shown great promise in helping make sense of EEG signals due to its capacity to learn good feature representations from raw data. Whether DL truly presents advantages as compared to more traditional EEG processing approaches, however, remains an open question. Objective In this work, we review 154 papers that apply DL to EEG, published between January 2010 and July 2018, and spanning different application domains such as epilepsy, sleep, brain-computer interfacing, and cognitive and affective monitoring. We extract trends and highlight interesting approaches from this large body of literature in order to inform future research and formulate recommendations. Methods Major databases spanning the fields of science and engineering were queried to identify relevant studies published in scientific journals, conferences, and electronic preprint repositories. Various data items were extracted for each study pertaining to (1) the data, (2) the preprocessing methodology, (3) the DL design choices, (4) the results, and (5) the reproducibility of the experiments. These items were then analyzed one by one to uncover trends. Results Our analysis reveals that the amount of EEG data used across studies varies from less than ten minutes to thousands of hours, while the number of samples seen during training by a network varies from a few dozens to several millions, depending on how epochs are extracted. Interestingly, we saw that more than half the studies used publicly available data and that there has also been a clear shift from intra-subject to inter-subject approaches over the last few years. About [Formula: see text] of the studies used convolutional neural networks (CNNs), while [Formula: see text] used recurrent neural networks (RNNs), most often with a total of 3-10 layers. Moreover, almost one-half of the studies trained their models on raw or preprocessed EEG time series. Finally, the median gain in accuracy of DL approaches over traditional baselines was [Formula: see text] across all relevant studies. More importantly, however, we noticed studies often suffer from poor reproducibility: a majority of papers would be hard or impossible to reproduce given the unavailability of their data and code. Significance To help the community progress and share work more effectively, we provide a list of recommendations for future studies and emphasize the need for more reproducible research. We also make our summary table of DL and EEG papers available and invite authors of published work to contribute to it directly. A planned follow-up to this work will be an online public benchmarking portal listing reproducible results.

https://iopscience.iop.org/article/10.1088/1741-2552/ab260c/pdf

Deep learning-based electroencephalography analysis: a systematic review.

Emotion recognition based on physiological signals has been a hot topic and applied in many areas such as safe driving, health care and social security. In this paper, we present a comprehensive review on physiological signal-based emotion recognition, including emotion models, emotion elicitation methods, the published emotional physiological datasets, features, classifiers, and the whole framework for emotion recognition based on the physiological signals. A summary and comparation among the recent studies has been conducted, which reveals the current existing problems and the future work has been discussed.

https://www.mdpi.com/1424-8220/18/7/2074/pdf

A Review of Emotion Recognition Using Physiological Signals

Communication without words

Recognition of emotions using multimodal physiological signals and an ensemble deep learning model

Automatic emotion recognition based on multi-channel neurophysiological signals, as a challenging pattern recognition task, is becoming an important computer-aided method for emotional disorder diagnoses in neurology and psychiatry. Traditional approaches require designing and extracting a range of features from single or multiple channel signals based on extensive domain knowledge. This may be an obstacle for non-domain experts. Moreover, traditional feature fusion method can not fully utilize correlation information between different channels. In this paper, we propose a preprocessing method that encapsulates the multi-channel neurophysiological signals into grid-like frames through wavelet and scalogram transform. We further design a hybrid deep learning model that combines the ‘Convolutional Neural Network (CNN)’ and ‘Recurrent Neural Network (RNN)’, for extracting task-related features, mining inter-channel correlation and incorporating contextual information from those frames. Experiments are carried out, in a trial-level emotion recognition task, on the DEAP benchmarking dataset. Our results demonstrate the effectiveness of the proposed methods, with respect to the emotional dimensions of Valence and Arousal.

Emotion recognition from multi-channel EEG data through Convolutional Recurrent Neural Network

Capturing user’s emotional state is an emerging way for implicit relevance feedback in information retrieval (IR). Recently, EEG-based emotion recognition has drawn increasing attention. However, a key challenge is effective learning of useful features from EEG signals. In this paper, we present our on-going work on using Deep Belief Network (DBN) to automatically extract high-level features from raw EEG signals. Our preliminary experiment on the DEAP dataset shows that the learned features perform comparably to the use of manually generated features for emotion recognition.

EEG Based Emotion Identification Using Unsupervised Deep Feature Learning

A Quantum-Like multimodal network framework for modeling interaction dynamics in multiparty conversational sentiment analysis

Affective state recognition based on multiple modalities of physiological signals has been a hot research topic. Traditional methods require designing hand-crafted features based on domain knowledge, which is time-consuming and has not achieved a satisfactory performance. On the other hand, conducting classification on raw signals directly can also cause some problems, such as the interference of noise and the curse of dimensionality. To address these problems, we propose a novel approach that encodes different modalities of data as images and use convolutional neural networks (CNN) to perform the affective state recognition task. We validate our aproach on the DECAF dataset in comparison with two state-of-the-art methods, i.e., the Support Vector Machines (SVM) and Random Forest (RF). Experimental results show that our aproach outperforms the baselines by 5% to 9%.

Encoding physiological signals as images for affective state recognition using convolutional neural networks

In recent years, “environmental, social, and governance” (ESG) stocks have become the best investment considerations. This article selects data from the ESG 300 Index (931463.CSI) and ESG 300 Index (399378.SZ) of the Chinese stock market and analyze its volatility and stock performance from April 2020 to September 2021. We compare the Sharpe Ratio, Sotino Ratio, Treynor Ratio, and Information Ratio of the ESG 300 Index with the Shanghai Stock Exchange and Shenzhen Stock Exchange Composite Index. We use GARCH (1,1) model to assess the volatility and apply machine learning technical skills, including KNN, SVM, and AdaBoost algorithm, to identify the relationship between ESG scores and stock returns collected from RANKING CSR Rating Agency. Based on different ESG scores, we construct hedge fund portfolios and separate them into four classes: Excellent ESG portfolios, Good ESG portfolios, Fair ESG portfolios, and Bad ESG portfolios, and compare with the Sharpe ratio under the situation of the effective tangent frontier. The results show that, compared with non-ESG-related stocks, ESG-related stocks cannot bring excess returns and have better risk performance during a normal period.

Guangliang Yu

Papers

Emotion recognition from multi-channel EEG data through Convolutional Recurrent Neural Network

EEG Based Emotion Identification Using Unsupervised Deep Feature Learning

A Quantum-Like multimodal network framework for modeling interaction dynamics in multiparty conversational sentiment analysis

Encoding physiological signals as images for affective state recognition using convolutional neural networks

Data Analysis of ESG Stocks in the Chinese Stock Market Based on Machine Learning