Brain computer interfaces rely on machine learning (ML) algorithms to decode the brain's electrical activity into decisions. For example, in rapid serial visual presentation (RSVP) tasks, the subject is presented with a continuous stream of images containing rare target images among standard images, while the algorithm has to detect brain activity associated with target images. Here, we continue our previous work, presenting a deep neural network model for the use of single trial EEG classification in RSVP tasks. Deep neural networks have shown state of the art performance in computer vision and speech recognition and thus have great promise for other learning tasks, like classification of EEG samples. In our model, we introduce a novel spatio-temporal regularization for EEG data to reduce overfitting. We show improved classification performance compared to our earlier work on a five categories RSVP experiment. In addition, we compare performance on data from different sessions and validate the model on a public benchmark data set of a P300 speller task. Finally, we discuss the advantages of using neural network models compared to manually designing feature extraction algorithms.

/pdf/convolutional-neural-network-for-multi-category-rapid-serial-bnzioh3s6d.pdf

Convolutional Neural Network for Multi-Category Rapid Serial Visual Presentation BCI.

Systematic comparison between a wireless EEG system with dry electrodes and a wired EEG system with wet electrodes.

Rapid serial visual presentation (RSVP) combined with the detection of event related brain responses facilitates the selection of relevant information contained in a stream of images presented rapidly to a human. Event related potentials (ERPs) measured non-invasively with electroencephalography (EEG) can be associated with infrequent targets amongst a stream of images. Human-machine symbiosis may be augmented by enabling human interaction with a computer, without overt movement, and/or enable optimization of image/information sorting processes involving humans. Features of the human visual system impact on the success of the RSVP paradigm, but pre-attentive processing supports the identification of target information post presentation of the information by assessing the co-occurrence or time-locked EEG potentials. This paper presents a comprehensive review and evaluation of the limited but significant literature on research in RSVP-based brain-computer interfaces (BCIs). Applications that use RSVP-based BCIs are categorized based on display mode and protocol design, whilst a range of factors influencing ERP evocation and detection are analyzed. Guidelines for using the RSVP-based BCI paradigms are recommended, with a view to further standardizing methods and enhancing the inter-relatability of experimental design to support future research and the use of RSVP-based BCIs in practice.

https://iopscience.iop.org/article/10.1088/1741-2552/aa9817/pdf

A review of rapid serial visual presentation-based brain-computer interfaces.

Ensemble deep learning for automated visual classification using EEG signals

Brain computer interfaces allow users to preform various tasks using only the electrical activity of the brain. BCI applications often present the user a set of stimuli and record the corresponding electrical response. The BCI algorithm will then have to decode the acquired brain response and perform the desired task. In rapid serial visual presentation (RSVP) tasks, the subject is presented with a continuous stream of images containing rare target images among standard images, while the algorithm has to detect brain activity associated with target images. In this work, we suggest a multimodal neural network for RSVP tasks. The network operates on the brain response and on the initiating stimulus simultaneously, providing more information for the BCI application. We present two variants of the multimodal network, a supervised model, for the case when the targets are known in advanced, and a semi-supervised model for when the targets are unknown. We test the neural networks with a RSVP experiment on satellite imagery carried out with two subjects. The multimodal networks achieve a significant performance improvement in classification metrics. We visualize what the networks has learned and discuss the advantages of using neural network models for BCI applications.

/pdf/multimodal-neural-network-for-rapid-serial-visual-37a8id10no.pdf

Multimodal Neural Network for Rapid Serial Visual Presentation Brain Computer Interface.

Brain computer interface applications, developed for both healthy and clinical populations, critically depend on decoding brain activity in single trials. The goal of the present study was to detect distinctive spatiotemporal brain patterns within a set of event related responses. We introduce a novel classification algorithm, the spatially weighted FLD-PCA (SWFP), which is based on a two-step linear classification of event-related responses, using fisher linear discriminant (FLD) classifier and principal component analysis (PCA) for dimensionality reduction. As a benchmark algorithm, we consider the hierarchical discriminant component Analysis (HDCA), introduced by Parra, et al. 2007. We also consider a modified version of the HDCA, namely the hierarchical discriminant principal component analysis algorithm (HDPCA). We compare single-trial classification accuracies of all the three algorithms, each applied to detect target images within a rapid serial visual presentation (RSVP, 10 Hz) of images from five different object categories, based on single-trial brain responses. We find a systematic superiority of our classification algorithm in the tested paradigm. Additionally, HDPCA significantly increases classification accuracies compared to the HDCA. Finally, we show that presenting several repetitions of the same image exemplars improve accuracy, and thus may be important in cases where high accuracy is crucial.

Spatiotemporal representations of rapid visual target detection: a single-trial EEG classification algorithm.

Systems and method for conduction of single trial classification of EEG signals of a human subject generated responsive to a series of images containing target images and non-target images, the method comprising: obtaining said EEG signals in a spatio-temporal representation comprising time points and respective spatial distribution of said EEG signals; classifying said time points independently, using a linear discriminant classifier, to compute spatio-temporal discriminating weights; using said spatio-temporal discriminating weights to amplify said spatio-temporal representation by said spatio-temporal discriminating weights at tempo-spatial points respectively, to create a spatially-weighted representation; using Principal Component Analysis (PCA) on a temporal domain for dimensionality reduction, separately for each spatial channel of said EEG signals, to create a PCA projection; applying said PCA projection to said spatially-weighted representation onto a first plurality of principal components, to create a temporally approximated spatially weighted representation containing for each spatial channel, PCA coefficients for said plurality of principal temporal projections; and classifying said temporally approximated spatially weighted representation, over said number of channels, using said linear discriminant classifier, to yield a binary decisions series indicative of each image of the images series as either belonging to said target image or to said non-target image.

Galit Fuhrmann Alpert

Papers

Spatiotemporal representations of rapid visual target detection: a single-trial EEG classification algorithm.

Classifying eeg signals in response to visual stimulus