The practical functionality of a brain-computer interface (BCI) is critically affected by the number of stimuli, especially for steady-state visual evoked potential based BCI (SSVEP-BCI), which shows promise for the implementation of a multi-target system for real-world applications. Joint frequency-phase modulation (JFPM) is an effective and widely used method in modulating SSVEPs. However, the ability of JFPM to implement an SSVEP-BCI system with a large number of stimuli, e.g., over 100 stimuli, remains unclear. To address this issue, a spectrally-dense JPFM (sJFPM) method is proposed to encode a broad array of stimuli, which modulates the low- and medium-frequency SSVEPs with a frequency interval of 0.1 Hz and triples the number of stimuli in conventional SSVEP-BCI to 120. To validate the effectiveness of the proposed 120-target BCI system, an offline experiment and a subsequent online experiment testing 18 healthy subjects in total were conducted. The offline experiment verified the feasibility of using sJFPM in designing an SSVEP-BCI system with 120 stimuli. Furthermore, the online experiment demonstrated that the proposed system achieved an average performance of 92.47±1.83% in online accuracy and 213.23±6.60 bits/min in online information transfer rate (ITR), where more than 75% of the subjects attained the accuracy above 90% and the ITR above 200 bits/min. This present study demonstrates the effectiveness of sJFPM in elevating the number of stimuli to more than 100 and extends our understanding of encoding a large number of stimuli by means of finer frequency division. 

/pdf/a-spectrally-dense-encoding-method-for-designing-a-high-1uxe11np.pdf

A Spectrally-Dense Encoding Method for Designing a High-Speed SSVEP-BCI With 120 Stimuli

Multiple types of brain-control systems have been applied in the field of rehabilitation. As an alternative scheme for balancing user fatigue and the classification accuracy of brain–computer interface (BCI) systems, facial-expression-based brain control technologies have been proposed in the form of novel BCI systems. Unfortunately, existing machine learning algorithms fail to identify the most relevant features of electroencephalogram signals, which further limits the performance of the classifiers. To address this problem, an improved classification method is proposed for facial-expression-based BCI (FE-BCI) systems, using a convolutional neural network (CNN) combined with a genetic algorithm (GA). The CNN was applied to extract features and classify them. The GA was used for hyperparameter selection to extract the most relevant parameters for classification. To validate the superiority of the proposed algorithm used in this study, various experimental performance results were systematically evaluated, and a trained CNN-GA model was constructed to control an intelligent car in real time. The average accuracy across all subjects was 89.21 ± 3.79%, and the highest accuracy was 97.71 ± 2.07%. The superior performance of the proposed algorithm was demonstrated through offline and online experiments. The experimental results demonstrate that our improved FE-BCI system outperforms the traditional methods.

https://www.frontiersin.org/articles/10.3389/fnins.2022.988535/pdf

A novel EEG decoding method for a facial-expression-based BCI system using the combined convolutional neural network and genetic algorithm

Analysis and application of functional connectivity in synchronic hybrid mental tasks for brain-computer interface

Background: The brain–computer interface (BCI) is a highly cross-discipline technology and its successful application in various domains has received increasing attention. However, the BCI-enabled automobile industry is has been comparatively less investigated. In particular, there are currently no studies focusing on brain-controlled driving mode selection. Specifically, different driving modes indicate different driving styles which can be selected according to the road condition or the preference of individual drivers. Methods: In this paper, a steady-state visual-evoked potential (SSVEP)-based driving mode selection system is proposed. Upon this system, drivers can select the intended driving modes by only gazing at the corresponding SSVEP stimuli. A novel EEG processing algorithm named inter-trial distance minimization analysis (ITDMA) is proposed to enhance SSVEP detection. Both offline and real-time experiments were carried out to validate the effectiveness of the proposed system. Conclusion: The results show that a high selection accuracy of up to 92.3% can be realized, although this depends on the specific choice of flickering duration, the number of EEG channels, and the number of training signals. Additionally, energy consumption is investigated in terms of which the proposed brain-controlled system considerably differs from a traditional driving mode selection system, and the main reason is shown to be the existence of a detection error.

/pdf/driving-mode-selection-through-ssvep-based-bci-and-energy-30ae9arz.pdf

Driving Mode Selection through SSVEP-Based BCI and Energy Consumption Analysis

The brain-computer interfaces (BCIs) based on steady-state visual evoked potential (SSVEP) have been extensively explored due to their advantages in terms of high communication speed and smaller calibration time. The visual stimuli in the low- and medium-frequency ranges are adopted in most of the existing studies for eliciting SSVEPs. However, there is a need to further improve the comfort of these systems. The high-frequency visual stimuli have been used to build BCI systems and are generally considered to significantly improve the visual comfort, but their performance is relatively low. The distinguishability of 16-class SSVEPs encoded by the three frequency ranges, i.e., 31-34.75 Hz with an interval of 0.25 Hz, 31-38.5 Hz with an interval of 0.5 Hz, 31-46 Hz with an interval of 1 Hz, is explored in this study. We compare classification accuracy and information transfer rate (ITR) of the corresponding BCI system. According to the optimized frequency range, this study builds an online 16-target high frequency SSVEP-BCI and verifies the feasibility of the proposed system based on 21 healthy subjects. The BCI based on visual stimuli with the narrowest frequency range, i.e., 31-34.5 Hz, have the highest ITR. Therefore, the narrowest frequency range is adopted to build an online BCI system. An averaged ITR obtained from the online experiment is 153.79 ±6.39 bits/min. These findings contribute to the development of more efficient and comfortable SSVEP-based BCIs. 

/pdf/optimizing-stimulus-frequency-ranges-for-building-a-high-11bbhzqx.pdf

Optimizing Stimulus Frequency Ranges for Building a High-Rate High Frequency SSVEP-BCI

Objective. Steady-state visual evoked potential (SSVEP)-brain-computer interfaces (BCIs) can cause much visual discomfort if the users use the SSVEP-BCIs for a long time. As an alternative scheme to reduce users' visual fatigue, this study proposes a new stimulation paradigm (termed as steady state peripheral visual evoked potential, abbreviated as SSPVEP) which makes full use of peripheral vision. The electroencephalography (EEG) signals are classifiable which means this proposed stimulation paradigm can be used in BCI system with the aid of the latest hybrid signal processing approach.Approach. Under the SSPVEP stimulation paradigm, 20 targets are mounted on 20 frequencies and other targets are set between two targets with flicker stimuli coding. In order to ensure the classification accuracy of SSPVEP signal detection under the proposed stimulation paradigm, two optimization schemes are proposed for the detection stage of the conventional ensemble task-related component analysis (ETRCA) algorithm. The first optimization scheme uses nonlinear correlation coefficient at the detection part for the first time to improve the classification accuracy of the system. The second optimization scheme usesγcorrection to enhance the time domain features of the SSPVEP signals, and uses Manhattan distance for the final detection.Main results. According to the response waveforms of the EEG signals generated under the SSPVEP stimulation paradigm and the results of the questionnaire on user's comfort level to the two stimulation paradigms (SSPVEP paradigm and conventional SSVEP paradigm), the proposed stimulation paradigm brings less visual fatigue. The comparison results indicate that the proposed detection methods (ETRCA +γcorrection + Manhattan distance, ETRCA + Spearman correlation) can greatly improve the classification accuracy compared with the individual template canonical correlation analysis method and conventional ETRCA method based on Pearson correlation.Significance. The SSPVEP stimulation paradigm reduces users' visual fatigue via using peripheral vision, which provides a new design idea for SSVEP stimulation paradigm aimed at visual comfort.

A comfortable steady state visual evoked potential stimulation paradigm using peripheral vision.

In recent years, deep neural network-based transfer learning (TL) has shown outstanding performance in EEG-based motor imagery (MI) brain-computer interface (BCI). However, due to the long preparation for pre-trained models and the arbitrariness of source domain selection, using deep transfer learning on different datasets and models is still challenging. In this paper, we proposed a multi-direction transfer learning (MDTL) strategy for cross-subject MI EEG-based BCI. This strategy utilizes data from multi-source domains to the target domain as well as from one multi-source domain to another multi-source domain. This strategy is model-independent so that it can be quickly deployed on existing models. Three generic deep learning models for MI classification (DeepConvNet, ShallowConvNet, and EEGNet) and two public motor imagery datasets (BCIC IV dataset 2a and Lee2019) are used in this study to verify the proposed strategy. For the four-classes dataset BCIC IV dataset 2a, the proposed MDTL achieves 80.86%, 81.95%, and 75.00% mean prediction accuracy using the three models, which outperforms those without MDTL by 5.79%, 6.64%, and 11.42%. For the binary-classes dataset Lee2019, MDTL achieves 88.2% mean accuracy using the model DeepConvNet. It outperforms the accuracy without MDTL by 23.48%. The achieved 81.95% and 88.2% are also better than the existing deep transfer learning strategy. Besides, the training time of MDTL is reduced by 93.94%. MDTL is an easy-to-deploy, scalable and reliable transfer learning strategy for existing deep learning models, which significantly improves model performance and reduces preparation time without changing model architecture.

https://ieeexplore.ieee.org/ielx7/7333/4359219/10077444.pdf

MDTL: A Novel and Model-Agnostic Transfer Learning Strategy for Cross-Subject Motor Imagery BCI

In steady-state visual-evoked potential (SSVEP) based brain-computer interfaces (BCIs), existing detection algorithms utilizing spatial filters like task-related component analysis (TRCA) derive the spatial filters mainly through maximizing the inter-trial similarity between the combined signals over the training set. Although they achieve by far the best classification performance in SSVEP-based BCIs, some important problems are still unresolved. Especially, the mechanism of how spatial filters cancel the background noise in brain signals and optimize the signal-to-noise ratio (SNR) of SSVEPs is still not figured out. Therefore, to solve these problems, in this paper a new perspective of spatial filter design is proposed. Specifically, a linear generative signal model of SSVEP is adopted and the spatial filters are obtained automatically through maximum likelihood estimation of source signals and channel vectors. In the same time, the relation between maximum likelihood estimation and signal-to-noise ratio (SNR) maximization is discussed. Through a step-by-step formulation, this paper provides a theoretical justification for those conventional algorithms utilizing spatial filters. As for the classification performance, the proposed scheme is tested on a benchmark dataset of 35 subjects. Experiment results show that the classification performance of the proposed scheme is competitive against three benchmark algorithms, which include TRCA. Especially, the proposed scheme achieves a fair performance improvement over the benchmark methods in the cases where a shorter time window, or a larger number of electrodes, or a smaller number of training blocks are adopted.

A Maximum Likelihood Perspective of Spatial Filter Design in SSVEP-Based BCIs

Brain-computer interface based on steady-state visual evoked potential (SSVEP-BCI) has been widely concerned because of its highest SNR of EEG signals. This kind of BCIs needs to modulate the control instructions with visual stimuli. However, a large number of visual stimuli will induce strong visual fatigue, and occupy a large area of screen space. As a result of the above disadvantages, the further application of this kind of BCIs can not be accepted by most users. It has been verified that 40 control instructions can be encoded with 20 flicker stimuli. The visual fatigue caused by SSVEP-BCIs can be alleviated by reducing the number of flicker stimuli. However, whether it is possible to encode the same number of control instructions with fewer flicker stimuli or not still remains unknown. This study will show the performance of the SSPVEP-BCIs after further removing some redundant flicker stimuli. We divide the proposed BCI system into four subsystems, and the performance of each subsystem will be shown in this paper. 12 flickering disks were used as visual stimuli to stimulate the brain to produce SSVEP responses. 12 targets were set on the flickering disks, and 28 targets were set alternately between the flicker stimuli, ensuring that always exist 1 to 2 flicker stimuli near each target. Manhattan distance based task-related component analysis (MTRCA) was used to process and decode the EEG signals in this paper. The performance within each of the subsystems was investigated and compared. When the sampling time was set to 4.6 s, the average accuracy of all subjects reaches 77.12%. This result shows that the 40-target BCI system based on proposed stimulation paradigm is available to the vast majority of users. Meanwhile, the accuracy of three subsystems is greater than 80% when the sampling time is 4.6 s. This result shows that the performance of SSPVEP stimulation paradigm is accepted in the application scenarios which only require a small number of instructions. In this paper, the research of SSPVEP paradigm is improved. It is revealed that using less flicker stimuli can still stimulate the brain to produce classifiable SSVEP responses. The conclusion of this study can guide the following study and application of SSPVEP-BCIs.

An SSPVEP Brain-Computer Interface Using a Small Amount of Flicker Stimuli

Objective. Visual evoked potentials (VEPs) have been commonly applied in brain-computer interfaces (BCIs) due to their satisfactory classification performance recently. However, most existing methods with flickering or oscillating stimuli will induce visual fatigue under long-term training, thus restricting the implementation of VEP-based BCIs. To address this issue, a novel paradigm adopting static motion illusion based on illusion-induced visual evoked potential is proposed for BCIs to enhance visual experience and practicality. Approach. This study explored the responses to baseline and illusion tasks including the rotating-tilted-lines illusion and rotating-snakes illusion. The distinguishable features were examined between different illusions by analyzing the event-related potentials and amplitude modulation of evoked oscillatory responses. Main results. The illusion stimuli elicited VEPs in an early time window encompassing a negative component (N1) from 110 to 200 ms and a positive component (P2) between 210 and 300 ms. Based on the feature analysis, a filter bank was designed to extract discriminative signals. The task-related component analysis was used to evaluate the binary classification task performance of the proposed method. Then the highest accuracy of 86.67% was achieved with a data length of 0.6 s. Significance. The results of this study demonstrate that the static motion illusion paradigm has the feasibility of implementation and is promising for VEP-based BCI applications.

Xi Zhao

Papers

A comfortable steady state visual evoked potential stimulation paradigm using peripheral vision.

MDTL: A Novel and Model-Agnostic Transfer Learning Strategy for Cross-Subject Motor Imagery BCI

A Maximum Likelihood Perspective of Spatial Filter Design in SSVEP-Based BCIs

An SSPVEP Brain-Computer Interface Using a Small Amount of Flicker Stimuli

A static paradigm based on illusion-induced VEP for brain-computer interfaces