Emotions have an important role in daily life, not only in human interaction, but also in decision-making processes, and in the perception of the world around us. Due to the recent interest shown by the research community in establishing emotional interactions between humans and computers, the identification of the emotional state of the former became a need. This can be achieved through multiple measures, such as subjective self-reports, autonomic and neurophysiological measurements. In the last years, Electroencephalography (EEG) received considerable attention from researchers, since it can provide a simple, cheap, portable, and ease-to-use solution for identifying emotions. In this paper, we present a survey of the neurophysiological research performed from 2009 to 2016, providing a comprehensive overview of the existing works in emotion recognition using EEG signals. We focus our analysis in the main aspects involved in the recognition process (e.g., subjects, features extracted, classifiers), and compare the works per them. From this analysis, we propose a set of good practice recommendations that researchers must follow to achieve reproducible, replicable, well-validated and high-quality results. We intend this survey to be useful for the research community working on emotion recognition through EEG signals, and in particular for those entering this field of research, since it offers a structured starting point.

Emotions Recognition Using EEG Signals: A Survey

This paper describes a new noninvasive brain-actuated wheelchair that relies on a P300 neurophysiological protocol and automated navigation. When in operation, the user faces a screen displaying a real-time virtual reconstruction of the scenario and concentrates on the location of the space to reach. A visual stimulation process elicits the neurological phenomenon, and the electroencephalogram (EEG) signal processing detects the target location. This location is transferred to the autonomous navigation system that drives the wheelchair to the desired location while avoiding collisions with obstacles in the environment detected by the laser scanner. This concept gives the user the flexibility to use the device in unknown and evolving scenarios. The prototype was validated with five healthy participants in three consecutive steps: screening (an analysis of three different groups of visual interface designs), virtual-environment driving, and driving sessions with the wheelchair. On the basis of the results, this paper reports the following evaluation studies: 1) a technical evaluation of the device and all functionalities; 2) a users' behavior study; and 3) a variability study. The overall result was that all the participants were able to successfully operate the device with relative ease, thus showing a great adaptation as well as a high robustness and low variability of the system.

/pdf/a-noninvasive-brain-actuated-wheelchair-based-on-a-p300-3q5nh90v4j.pdf

A Noninvasive Brain-Actuated Wheelchair Based on a P300 Neurophysiological Protocol and Automated Navigation

Significant progress has been made towards the clinical application of human-machine interfaces (HMIs) based on electromyographic (EMG) pattern recognition for various rehabilitation purposes. Making this technology practical and available to patients with motor deficits requires overcoming real-world challenges, such as physical and physiological changes, that result in variations in EMG signals and systems that are unreliable for long-term use. In this study, we aimed to address these challenges by (1) investigating the stability of time-domain EMG features during changes in the EMG signals and (2) identifying the feature sets that would provide the most robust EMG pattern recognition. Variations in EMG signals were introduced during physical experiments. We identified three disturbances that commonly affect EMG signals: EMG electrode location shift, variation in muscle contraction effort, and muscle fatigue. The impact of these disturbances on individual features and combined feature sets was quantified by changes in classification performance. The robustness of feature sets was evaluated by a stability index developed in this study. Muscle fatigue had the smallest effect on the studied EMG features, while electrode location shift and varying effort level significantly reduced the classification accuracy for most of the features. Under these disturbances, the most stable EMG feature set with combination of four features produced at least 16.0% higher classification accuracy than the least stable set. EMG autoregression coefficients and cepstrum coefficients showed the most robust classification performance of all studied time-domain features. Selecting appropriate EMG feature combinations can overcome the impact of the studied disturbances on EMG pattern classification to a certain extent; however, this simple solution is still inadequate. Stabilizing electrode contact locations and developing effective classifier training strategies are suggested to further improve the robustness of HMIs based on EMG pattern recognition.

/pdf/study-of-stability-of-time-domain-features-for-1gzo6mu6zo.pdf

Study of stability of time-domain features for electromyographic pattern recognition

EEG-based brain-controlled mobile robots can serve as powerful aids for severely disabled people in their daily life, especially to help them move voluntarily. In this paper, we provide a comprehensive review of the complete systems, key techniques, and evaluation issues of brain-controlled mobile robots along with some insights into related future research and development issues. We first review and classify various complete systems of brain-controlled mobile robots into two categories from the perspective of their operational modes. We then describe key techniques that are used in these brain-controlled mobile robots including the brain-computer interface techniques and shared control techniques. This description is followed by an analysis of the evaluation issues of brain-controlled mobile robots including participants, tasks and environments, and evaluation metrics. We conclude this paper with a discussion of the current challenges and future research directions.

EEG-Based Brain-Controlled Mobile Robots: A Survey

In this research, emotional states in arousal/valence dimensions have been classified using minimum number of channels and frequency bands of EEG signal. Using the discrete wavelet transforms, EEG signals have been decomposed to corresponding frequency bands and then several features have been extracted. The support vector machine and K-nearest neighbor classifiers have been used to detect the emotional states from the extracted features. For the recorded 10-channel EEG signal, results illustrate the classification accuracy of 86.75 % for arousal level and 84.05 % for valence level. Moreover, using the high-frequency bands, specifically gamma band, yields higher accuracy compared to using low-frequency bands of EEG signal. All of these support to the development of a real-time emotion classification system.

Wavelet-based emotion recognition system using EEG signal

Two different Human-Machine Interfaces (HMIs) were developed, both based on electro-biological signals. One is based on the EMG signal and the other is based on the EEG signal. Two major features of such interfaces are their relatively simple data acquisition and processing systems, which need just a few hardware and software resources, so that they are, computationally and financially speaking, low cost solutions. Both interfaces were applied to robotic systems, and their performances are analyzed here. The EMG-based HMI was tested in a mobile robot, while the EEG-based HMI was tested in a mobile robot and a robotic manipulator as well. Experiments using the EMG-based HMI were carried out by eight individuals, who were asked to accomplish ten eye blinks with each eye, in order to test the eye blink detection algorithm. An average rightness rate of about 95% reached by individuals with the ability to blink both eyes allowed to conclude that the system could be used to command devices. Experiments with EEG consisted of inviting 25 people (some of them had suffered cases of meningitis and epilepsy) to test the system. All of them managed to deal with the HMI in only one training session. Most of them learnt how to use such HMI in less than 15 minutes. The minimum and maximum training times observed were 3 and 50 minutes, respectively. Such works are the initial parts of a system to help people with neuromotor diseases, including those with severe dysfunctions. The next steps are to convert a commercial wheelchair in an autonomous mobile vehicle; to implement the HMI onboard the autonomous wheelchair thus obtained to assist people with motor diseases, and to explore the potentiality of EEG signals, making the EEG-based HMI more robust and faster, aiming at using it to help individuals with severe motor dysfunctions.

/pdf/human-machine-interfaces-based-on-emg-and-eeg-applied-to-2i4oix8g5l.pdf

Human-machine interfaces based on EMG and EEG applied to robotic systems

We present in this paper a study of three EEG signals feature extraction techniques. These techniques have been widely employed in researches of emotional states recognition: statistical characteristics, features based on PSD (Power Spectral Density) and features based on HOC (High Order Crossings). The validation was performed via classification of emotional states of calm and stress using the K-NN based classifier in off-line mode using EEG signals from available DEAP database. The best results achieved were 70.1%, using the PSD based technique, and 69.59% using the HOC based technique.

Evaluation of feature extraction techniques in emotional state recognition

This paper presents a Human-Machine Interface (HMI) based on the signals generated by eye blinks or brain activity. The system structure and the signal acquisition and processing are shown. The signals used in this work are either the signal associated to the muscular movement corresponding to an eye blink or the brain signal corresponding to visual information processing. The variance is the feature extracted from such signals in order to detect the intention of the user. The classification is performed by a variance threshold which is experimentally determined for each user during the training stage. The command options, which are going to be sent to the commanded device, are presented to the user in the screen of a PDA (Personal Digital Assistant). In the experiments here reported, a robotic wheelchair is used as the device being commanded.

https://iopscience.iop.org/article/10.1088/1742-6596/90/1/012094/pdf

Human–machine interface based on muscular and brain signals applied to a robotic wheelchair

IntroductionThe main idea of a traditional Steady State Visually Evoked Potentials (SSVEP)-BCI is the activation of commands through gaze control. For this purpose, the retina of the eye is excited by a stimulus at a certain frequency. Several studies have shown effects related to different kind of stimuli, frequencies, window lengths, techniques of feature extraction and even classification. So far, none of the previous studies has performed a comparison of performance of stimuli colors through LED technology. This study addresses precisely this important aspect and would be a great contribution to the topic of SSVEP-BCIs. Additionally, the performance of different colors at different frequencies and the visual comfort were evaluated in each case.MethodsLEDs of four different colors (red, green, blue and yellow) flickering at four distinct frequencies (8, 11, 13 and 15 Hz) were used. Twenty subjects were distributed in two groups performing different protocols. Multivariate Synchronization Index (MSI) was the technique adopted as feature extractor.ResultsThe accuracy was gradually enhanced with the increase of the time window. From our observations, the red color provides, in most frequencies, both highest rates of accuracy and Information Transfer Rate (ITR) for detection of SSVEP.ConclusionAlthough the red color has presented higher ITR, this color was turned in the less comfortable one and can even elicit epileptic responses according to the literature. For this reason, the green color is suggested as the best choice according to the proposed rules. In addition, this color has shown to be safe and accurate for an SSVEP-BCI.

/pdf/comparison-of-the-influence-of-stimuli-color-on-steady-state-hyide9flxs.pdf

Comparison of the influence of stimuli color on Steady-State Visual Evoked Potentials

This paper presents the evaluation of seven techniques of feature extraction (PSD, F-Test, EMD, MCE, CCA, LASSO and MSI) for gaze-target detections in a SSVEP-based BCI. Two type of technologies for visual stimulation were used (LCD and LEDs). Five differents windows lengths (1, 2, 4, 5 and 10 s) were used and seven volunteers participated in this study. The highest accuracy obtained in all cases was 93.57% using LEDs and the highest ITR was 36.90 bits/min for LCD. The technique based on MSI shows the highest success rate in both cases (LCD or LED) and is even more noticeable when the window size is increased.

André Ferreira

Papers

Human-machine interfaces based on EMG and EEG applied to robotic systems

Evaluation of feature extraction techniques in emotional state recognition

Human–machine interface based on muscular and brain signals applied to a robotic wheelchair

Comparison of the influence of stimuli color on Steady-State Visual Evoked Potentials

A comparison of techniques and technologies for SSVEP classification