Recent advances in brain-computer interfaces
Summary (5 min read)
Introduction
- The ability to communicate with other persons, be it through speech, gesturing, or writing, is one of the main factors making the life of any human being enjoyable.
- A disease that is known to lead to the locked-in syndrome is amyotrophic lateral sclerosis (ALS), also known as Lou Gehrig’s disease.
- Beyond communication, other applications of BCI involving multimedia can also be envisioned.
- In the second part of the paper (Section V) a concrete state-of-the-art BCI system is briefly described.
II. NEUROPHYSIOLOGIC SIGNALS
- To control a BCI, users have to acquire conscious control over their brain activity.
- Two fundamentally different approaches exist to achieve this.
- In the first approach, subjects perceive a set of stimuli displayed by the BCI system and can control their brain activity by focusing onto one specific stimulus.
- The changes in neurophysiologic signals resulting from perception and processing of stimuli are termed event-related potentials (ERPs) and are discussed together with the corresponding BCI paradigms in subsection II-A.
- In this approach feedback signals are often used to let subjects learn the production of easily detectable patterns of neurophysiologic signals.
B. Oscillatory Brain Activity
- Sinusoid like oscillatory brain activity occurs in many regions of the brain and changes its characteristics according to the state of subjects, for example between wake and sleep or between concentrated work and idling.
- Oscillatory activity in the EEG is classified into different frequency bands or rhythms.
- The changes in the mu-rhythm are localized over the part of the sensorimotor cortex corresponding to the body part, and so imagined movements of different body parts can be discriminated.
- Since the changes in mu- rhythm occurring in untrained users are usually not strong enough to be detected by a classification algorithm, feedback training has to be used.
- BCI systems employing imagined movements of hands, feet, or tongue have been mainly introduced by the research group of Pfurtscheller in Austria [9].
C. Slow Cortical Potentials
- Slow cortical potentials (SCPs) are slow voltage shifts in the EEG occurring in the frequency range 1-2 Hz.
- Through feedback training subjects can learn to voluntarily control their SCPs.
- The voluntary production of negative and positive SCPs has been exploited by Birbaumer et al to show that patients suffering from ALS can use a BCI to control a spelling device and to communicate with their environment [11].
D. Neuronal Ensemble Activity
- Action potentials are thought to be the basic unit of information in the brain and enable communication between different neurons.
- For example the firing rate of ensembles of neurons in the motor and premotor-cortices can be used to predict hand positions or hand velocities.
- Neuronal ensemble activity can thus be employed as neurophysiological signal in BCIs, in particular in BCIs using microelectrode arrays [12].
- 1The term mu-rhythm is used for oscillatory activity with a frequency of about 10 Hz, localized over the sensorimotor cortex.
- The term alpha-rhythm is more general and can be used for any activity in the frequency range 8 - 13 Hz.
III. SIGNAL PROCESSING AND MACHINE LEARNING
- In the previous section the authors have discussed paradigms that let users control their brain activity and the neurophysiologic signals corresponding to the respective paradigms.
- In BCIs, machine learning algorithms are applied to learn from a training dataset how to classify the signals of a specific user.
- As is well known, most machine learning algorithms can be divided into two modules: signal processing (also known as feature extraction) and classification.
- In the following subsections the authors first review signal processing methods that are typically used in BCIs and then give a short introduction to classification methods for BCIs.
A. Time Domain Features
- Good examples for signals that can be characterized with the help of time domain features are the P300 and SCPs.
- A strategy that is often used to separate these signals from background activity and noise is lowpass or bandpass filtering, optionally followed by downsampling.
- Systems based on the discrete wavelet transform (DWT), as well as systems based on the continuous wavelet transform (CWT) have been described in the literature.
- Besides the use for the EEG signals P300, SCP, and MRP, time domain features are also used in BCI systems based on neuronal ensemble activity.
- These techniques will however not be further discussed here.
B. Frequency Domain Features
- Frequency domain features are related to changes in oscillatory activity.
- The most commonly used frequency domain features are related to changes in the amplitude of oscillatory activity.
- To estimate band power, different methods have been used.
- These include Welch’s method [7], adaptive autoregressive models [13], and Morlet wavelets [14].
- Synchronization of signals from different brain regions might indicate that these regions communicate.
C. Spatial Domain Features
- The feature extraction techniques described so far all work with univariate time series, i.e. data from only one electrode is used (an exception are synchronization features, extracted from bivariate time series).
- In many systems however, data from more than one electrode is available.
- Finding efficient combinations of features from more than one electrode is the goal of spatial feature extraction methods.
- Another method for computing the coefficients of spatial filters from training data is independent component analysis (ICA).
- ICA has been mainly used in P300-based BCIs as a feature extraction method (see for example [19]).
A. Spelling Devices
- Spelling devices allow severely disabled users to communicate with their environment by sequentially selecting symbols from the alphabet.
- One of the first spelling devices mentioned in the BCI literature is the P300 speller [6].
- Another system, tested with users suffering from ALS and based on SCPs was described by Birbaumer [11].
C. Wheelchair Control
- Because steering a wheelchair is a complex task and because wheelchair control has to be extremely reliable, the possible movements of the wheelchair are strongly constrained in current prototype systems.
- The wheelchair is constrained to move along paths predefined in software, joining registered locations, and a P300-based interface is used to select the desired location.
E. Gaming and Virtual Reality
- Besides the applications targeted towards disabled subjects, prototypes of gaming and virtual reality applications have been described in the literature.
- Examples for such applications are the control of a spaceship with oscillatory brain activity [28] and the control of an animated character in an immersive 3D gaming environment with SSVEPs [7].
V. AN EFFICIENT BRAIN-COMPUTER INTERFACE FOR DISABLED SUBJECTS
- After the general review of neurophysiologic signals, signal processing and machine learning methods, and BCI applications, the authors now turn their attention to a more detailed description of a state-of-the-art BCI system.
- More specifically, a BCI system using the P300, developed in the Multimedia Signal Processing Group at the EPFL, is described.
- The interested reader can find more details about this system in [29] and [30].
A. System Description
- In the BCI system developed at the EPFL users are facing a laptop screen on which six images are displayed (see Fig. 2).
- The application scenario serves however only as an example and is not pursued in further detail.
- The images are flashed in random sequences, one image at a time.
- The stimulus display and the online access to the EEG signals are implemented as dynamic link libraries (DLLs) in C.
- The DLLs are accessed from MATLAB via a MEX interface.
B. Materials and Methods
- The system is tested with five disabled and four able-bodied subjects.
- The able-bodied subjects are PhD students recruited from their laboratory.
- For all subjects the time between the first and the last session is less than two weeks.
- Each of the sessions consists of six runs, one run for each of the six images.
- Is inferred from the EEG with a simple classifier2.
C. Signal Processing and Machine Learning
- To extract features from the raw EEG signals, the following operations are applied: 1) Referencing 2) Filtering A 6th order forward-backward Butterworth bandpass filter is used to filter the data.
- The samples from the selected electrodes are concatenated into feature vectors.
- Note that the posterior distribution depends on the hyperparameters α and β.
- The Bayesian regression framework offers an elegant and computationally efficient solution for the problem of choosing the hyperparameters.
D. Results
- To give an idea of the performance that can be achieved with their BCI system, the authors have plotted the classification accuracy and the bitrate for one disabled subject in Fig.
- This procedure was repeated four times, such that each of the sessions served once as testing session.
- The maximal bitrate, computed according to the definition of Wolpaw [3], was approximately 25 bits/min.
- For three of the other four disabled subjects tested in their study 100% classification accuracy was also achieved and the maximal bitrate varied between 9 and 19 bits/min.
- This might be explained by the fact that the level of alertness of the subject fluctuated strongly and rapidly during the experiments.
A. Asynchronous BCI
- One significant limitation of the P300-based BCI presented in this paper and of many other BCI systems is that they only work in synchronous mode.
- This means that users can only communicate via the BCI at time instants predetermined by the system and that the system has to be switched on/off by a caretaker.
- Asynchronous BCI systems can detect autonomously that a user is trying to communicate via the BCI and hence allow for more realistic application scenarios than synchronous systems.
- First, experimental protocols and evaluation criteria for asynchronous BCI systems should be defined.
- Second, algorithms that can detect if the user wants to communicate via the BCI or is engaged in other activity have to be developed.
B. Using a BCI Without Training
- In almost all current BCI systems, subjects first have to go through a training phase, in which they concentrate on prescribed mental tasks or prescribed stimuli.
- A drawback of this setup is that for many disabled users a long training phase is an insurmountable obstacle due to cognitive impairments and concentration problems.
- Another problem is caused by the fact that patterns of cerebral activity are constantly changing, and hence new training sessions have to be performed periodically to adapt classification rules.
- One approach to overcome these problems, is to develop machine learning algorithms, with which subjects can immediately start using a BCI, without training.
- A class of machine learning algorithms that might be used to build such a classifier are so-called mixture of experts models [37].
VII. CONCLUSION
- Signal processing and machine learning methods, as well as applications for BCIs.the authors.
- One of the main features of this system is that it employs advanced Bayesian machine learning tools which makes training of classifiers simple, fast, and reliable.
- None of the systems described in the scientific literature is suited for daily use by disabled persons or for use in multimedia environments.
- This is due to the fact that the technology underlying BCIs is not yet mature enough for usage out of the laboratory.
- Many challenging and interesting questions in BCI research are thus still waiting to be explored.
Did you find this useful? Give us your feedback
Citations
14 citations
14 citations
Cites background or methods from "Recent advances in brain-computer i..."
...In the first method, the external stimuli cause changes in neurophysiologic signals called event-related potentials (ERPs) [15, 22] which are used to identify a user’s response to the stimuli presented....
[...]
...The idea underlying BCIs is to measure electric, magnetic, or other physical manifestations of the brain activity and to translate these into commands for a computer or other devices [21, 15]....
[...]
...For example, imagination of hand movement can be used to modify activity in the motor cortex [15]....
[...]
12 citations
12 citations
11 citations
References
19,261 citations
18,802 citations
6,803 citations
"Recent advances in brain-computer i..." refers background or methods in this paper
...The maximal bitrate, computed according to the definition of Wolpaw [3], was approximately 25 bits/min....
[...]
...Other reviews can be found in [2], [3] and [4]....
[...]
...The group of Wolpaw in the United States has also worked on such systems, and an impressive sensorimotor rhythm BCI allowing for fast control of a 2D cursor has been described in [10]....
[...]
4,194 citations
"Recent advances in brain-computer i..." refers background in this paper
...The maximum likelihood solution for the hyperparameters can be found with a simple iterative algorithm which we do not discuss in detail here but which is described in [29], [30], [34]....
[...]
3,120 citations
"Recent advances in brain-computer i..." refers methods in this paper
...Neuronal ensemble activity can thus be employed as neurophysiological signal in BCIs, in particular in BCIs using microelectrode arrays [12]....
[...]
Related Papers (5)
Frequently Asked Questions (19)
Q2. What are the main advantages of the SVM?
The main advantages of the SVM are that it allows to achieve very good classification accuracy and that nonlinear classification functions can be easily implemented by using kernels.
Q3. What is the common strategy used to separate the neurophysiologic signals from background activity?
A strategy that is often used to separate these signals from background activity and noise is lowpass or bandpass filtering, optionally followed by downsampling.
Q4. How many electrodes are used to record the EEG?
The EEG is recorded at 2048 Hz sampling rate from thirty-two electrodes placed at the standard positions of the 10-20 international system.
Q5. What are examples of ERPs that can be used in BCIs?
Other examples for ERPs that can be used in BCIs are steady-state visual evoked potentialss (SSVEPs) and motorrelated potentials (MRPs).
Q6. What is the assumption underlying the application of ICA to EEG signals?
The assumption underlying the application of ICA to EEG signals is that the signals measured on the scalp are a linear and instantaneous mixture of signals from independent sources in the cortex, deeper brain structures, and noise [18].
Q7. What is the drawback of training SVMs?
A drawback is however, that training SVMs is computationally complex because regularization constants and kernel parameters are typically estimated with a cross-validation procedure.
Q8. What are the main advantages of FDA?
The main advantages of FDA are that it is a computationally and conceptually simple method and that very good classification accuracy can be achieved.
Q9. What are the common time domain features used in BCIs?
Besides the use for the EEG signals P300, SCP, and MRP, time domain features are also used in BCI systems based on neuronal ensemble activity.
Q10. What is the definition of a Bayesian analysis?
Through a Bayesian analysis the degree of regularization can be estimated automatically and quickly from training data without the need for time consuming cross-validation.
Q11. What is the idea behind the use of a BCI?
The idea underlying research on neuromotor prostheses is to use a BCI for controlling movement of limbs and to restore motor function in tetraplegics or amputees.
Q12. What is the way to overcome these problems?
One approach to overcome these problems, is to develop machine learning algorithms, with which subjects can immediately start using a BCI, without training.
Q13. What is the common use of neuronal ensemble activity in BCIs?
Neuronal ensemble activity can thus be employed as neurophysiological signal in BCIs, in particular in BCIs using microelectrode arrays [12].
Q14. Why is the steering of a wheelchair a complex task?
Because steering a wheelchair is a complex task and becausewheelchair control has to be extremely reliable, the possible movements of the wheelchair are strongly constrained in current prototype systems.
Q15. What is the main disadvantage of the SVM?
A second issue is that the loss function used in the SVM is designed for problems in which only binary yes/no outputs are needed.
Q16. Why did the able-bodied subjects achieve better results than the disabled subjects?
While due to fatigue or concentration problems not all ablebodied subjects achieved 100% classification accuracy, the bitrates for the able-bodied subjects were in general higher than those of the disabled subjects.
Q17. What are the types of brain signals that are used in BCIs?
The types of signals resulting from concentration on mental tasks together with the corresponding BCI paradigms are described in subsections IIB, II-C, and II-D.ERPs are stereotyped, spatio-temporal patterns of brain activity, occurring time-locked to an event, for example after presentation of a stimulus, before execution of a movement, or after the detection of a novel stimulus.
Q18. What are the steps to develop asynchronous BCI systems?
algorithms that can detect if the user wants to communicate via the BCI or is engaged in other activity have to be developed.
Q19. How many disabled subjects were tested in their study?
For three of the other four disabled subjects tested in their study 100% classification accuracy was also achieved and the maximal bitrate varied between 9 and 19 bits/min.