scispace - formally typeset
Search or ask a question
Proceedings ArticleDOI

Recent advances in brain-computer interfaces

01 Oct 2007-pp 17-17
TL;DR: An overview of the aspects of BCI research mentioned above are given and recent developments and open problems are highlighted.
Abstract: A brain-computer interface (BCI) is a communication system that translates brain activity into commands for a computer or other devices. In other words, a BCI allows users to act on their environment by using only brain activity, without using peripheral nerves and muscles. The major goal of BCI research is to develop systems that allow disabled users to communicate with other persons, to control artificial limbs, or to control their environment. To achieve this goal, many aspects of BCI systems are currently being investigated. Research areas include evaluation of invasive and noninvasive technologies to measure brain activity, evaluation of control signals (i.e. patterns of brain activity that can be used for communication), development of algorithms for translation of brain signals into computer commands, and the development of new BCI applications. In this paper we give an overview of the aspects of BCI research mentioned above and highlight recent developments and open problems.

Summary (5 min read)

Introduction

  • The ability to communicate with other persons, be it through speech, gesturing, or writing, is one of the main factors making the life of any human being enjoyable.
  • A disease that is known to lead to the locked-in syndrome is amyotrophic lateral sclerosis (ALS), also known as Lou Gehrig’s disease.
  • Beyond communication, other applications of BCI involving multimedia can also be envisioned.
  • In the second part of the paper (Section V) a concrete state-of-the-art BCI system is briefly described.

II. NEUROPHYSIOLOGIC SIGNALS

  • To control a BCI, users have to acquire conscious control over their brain activity.
  • Two fundamentally different approaches exist to achieve this.
  • In the first approach, subjects perceive a set of stimuli displayed by the BCI system and can control their brain activity by focusing onto one specific stimulus.
  • The changes in neurophysiologic signals resulting from perception and processing of stimuli are termed event-related potentials (ERPs) and are discussed together with the corresponding BCI paradigms in subsection II-A.
  • In this approach feedback signals are often used to let subjects learn the production of easily detectable patterns of neurophysiologic signals.

A. Event-Related Potentials

  • The P300 is a positive deflection in the EEG, appearing approximately 300 ms after the presentation of rare or surprising, task-relevant stimuli [5].
  • To evoke the P300, subjects are asked to observe a random sequence of two types of stimuli.
  • One stimulus type (the oddball or target stimulus) appears only rarely in the sequence, while the other stimulus type (the normal or nontarget stimulus) appears more often.
  • This principle was exploited by Farwell and Donchin in a BCI system which allowed to spell words by sequentially selecting symbols from a matrix of symbols [6].
  • Descriptions of systems using such signals can be found in [7] and [8].

B. Oscillatory Brain Activity

  • Sinusoid like oscillatory brain activity occurs in many regions of the brain and changes its characteristics according to the state of subjects, for example between wake and sleep or between concentrated work and idling.
  • Oscillatory activity in the EEG is classified into different frequency bands or rhythms.
  • The changes in the mu-rhythm are localized over the part of the sensorimotor cortex corresponding to the body part, and so imagined movements of different body parts can be discriminated.
  • Since the changes in mu- rhythm occurring in untrained users are usually not strong enough to be detected by a classification algorithm, feedback training has to be used.
  • BCI systems employing imagined movements of hands, feet, or tongue have been mainly introduced by the research group of Pfurtscheller in Austria [9].

C. Slow Cortical Potentials

  • Slow cortical potentials (SCPs) are slow voltage shifts in the EEG occurring in the frequency range 1-2 Hz.
  • Through feedback training subjects can learn to voluntarily control their SCPs.
  • The voluntary production of negative and positive SCPs has been exploited by Birbaumer et al to show that patients suffering from ALS can use a BCI to control a spelling device and to communicate with their environment [11].

D. Neuronal Ensemble Activity

  • Action potentials are thought to be the basic unit of information in the brain and enable communication between different neurons.
  • For example the firing rate of ensembles of neurons in the motor and premotor-cortices can be used to predict hand positions or hand velocities.
  • Neuronal ensemble activity can thus be employed as neurophysiological signal in BCIs, in particular in BCIs using microelectrode arrays [12].
  • 1The term mu-rhythm is used for oscillatory activity with a frequency of about 10 Hz, localized over the sensorimotor cortex.
  • The term alpha-rhythm is more general and can be used for any activity in the frequency range 8 - 13 Hz.

III. SIGNAL PROCESSING AND MACHINE LEARNING

  • In the previous section the authors have discussed paradigms that let users control their brain activity and the neurophysiologic signals corresponding to the respective paradigms.
  • In BCIs, machine learning algorithms are applied to learn from a training dataset how to classify the signals of a specific user.
  • As is well known, most machine learning algorithms can be divided into two modules: signal processing (also known as feature extraction) and classification.
  • In the following subsections the authors first review signal processing methods that are typically used in BCIs and then give a short introduction to classification methods for BCIs.

A. Time Domain Features

  • Good examples for signals that can be characterized with the help of time domain features are the P300 and SCPs.
  • A strategy that is often used to separate these signals from background activity and noise is lowpass or bandpass filtering, optionally followed by downsampling.
  • Systems based on the discrete wavelet transform (DWT), as well as systems based on the continuous wavelet transform (CWT) have been described in the literature.
  • Besides the use for the EEG signals P300, SCP, and MRP, time domain features are also used in BCI systems based on neuronal ensemble activity.
  • These techniques will however not be further discussed here.

B. Frequency Domain Features

  • Frequency domain features are related to changes in oscillatory activity.
  • The most commonly used frequency domain features are related to changes in the amplitude of oscillatory activity.
  • To estimate band power, different methods have been used.
  • These include Welch’s method [7], adaptive autoregressive models [13], and Morlet wavelets [14].
  • Synchronization of signals from different brain regions might indicate that these regions communicate.

C. Spatial Domain Features

  • The feature extraction techniques described so far all work with univariate time series, i.e. data from only one electrode is used (an exception are synchronization features, extracted from bivariate time series).
  • In many systems however, data from more than one electrode is available.
  • Finding efficient combinations of features from more than one electrode is the goal of spatial feature extraction methods.
  • Another method for computing the coefficients of spatial filters from training data is independent component analysis (ICA).
  • ICA has been mainly used in P300-based BCIs as a feature extraction method (see for example [19]).

A. Spelling Devices

  • Spelling devices allow severely disabled users to communicate with their environment by sequentially selecting symbols from the alphabet.
  • One of the first spelling devices mentioned in the BCI literature is the P300 speller [6].
  • Another system, tested with users suffering from ALS and based on SCPs was described by Birbaumer [11].

C. Wheelchair Control

  • Because steering a wheelchair is a complex task and because wheelchair control has to be extremely reliable, the possible movements of the wheelchair are strongly constrained in current prototype systems.
  • The wheelchair is constrained to move along paths predefined in software, joining registered locations, and a P300-based interface is used to select the desired location.

E. Gaming and Virtual Reality

  • Besides the applications targeted towards disabled subjects, prototypes of gaming and virtual reality applications have been described in the literature.
  • Examples for such applications are the control of a spaceship with oscillatory brain activity [28] and the control of an animated character in an immersive 3D gaming environment with SSVEPs [7].

V. AN EFFICIENT BRAIN-COMPUTER INTERFACE FOR DISABLED SUBJECTS

  • After the general review of neurophysiologic signals, signal processing and machine learning methods, and BCI applications, the authors now turn their attention to a more detailed description of a state-of-the-art BCI system.
  • More specifically, a BCI system using the P300, developed in the Multimedia Signal Processing Group at the EPFL, is described.
  • The interested reader can find more details about this system in [29] and [30].

A. System Description

  • In the BCI system developed at the EPFL users are facing a laptop screen on which six images are displayed (see Fig. 2).
  • The application scenario serves however only as an example and is not pursued in further detail.
  • The images are flashed in random sequences, one image at a time.
  • The stimulus display and the online access to the EEG signals are implemented as dynamic link libraries (DLLs) in C.
  • The DLLs are accessed from MATLAB via a MEX interface.

B. Materials and Methods

  • The system is tested with five disabled and four able-bodied subjects.
  • The able-bodied subjects are PhD students recruited from their laboratory.
  • For all subjects the time between the first and the last session is less than two weeks.
  • Each of the sessions consists of six runs, one run for each of the six images.
  • Is inferred from the EEG with a simple classifier2.

C. Signal Processing and Machine Learning

  • To extract features from the raw EEG signals, the following operations are applied: 1) Referencing 2) Filtering A 6th order forward-backward Butterworth bandpass filter is used to filter the data.
  • The samples from the selected electrodes are concatenated into feature vectors.
  • Note that the posterior distribution depends on the hyperparameters α and β.
  • The Bayesian regression framework offers an elegant and computationally efficient solution for the problem of choosing the hyperparameters.

D. Results

  • To give an idea of the performance that can be achieved with their BCI system, the authors have plotted the classification accuracy and the bitrate for one disabled subject in Fig.
  • This procedure was repeated four times, such that each of the sessions served once as testing session.
  • The maximal bitrate, computed according to the definition of Wolpaw [3], was approximately 25 bits/min.
  • For three of the other four disabled subjects tested in their study 100% classification accuracy was also achieved and the maximal bitrate varied between 9 and 19 bits/min.
  • This might be explained by the fact that the level of alertness of the subject fluctuated strongly and rapidly during the experiments.

A. Asynchronous BCI

  • One significant limitation of the P300-based BCI presented in this paper and of many other BCI systems is that they only work in synchronous mode.
  • This means that users can only communicate via the BCI at time instants predetermined by the system and that the system has to be switched on/off by a caretaker.
  • Asynchronous BCI systems can detect autonomously that a user is trying to communicate via the BCI and hence allow for more realistic application scenarios than synchronous systems.
  • First, experimental protocols and evaluation criteria for asynchronous BCI systems should be defined.
  • Second, algorithms that can detect if the user wants to communicate via the BCI or is engaged in other activity have to be developed.

B. Using a BCI Without Training

  • In almost all current BCI systems, subjects first have to go through a training phase, in which they concentrate on prescribed mental tasks or prescribed stimuli.
  • A drawback of this setup is that for many disabled users a long training phase is an insurmountable obstacle due to cognitive impairments and concentration problems.
  • Another problem is caused by the fact that patterns of cerebral activity are constantly changing, and hence new training sessions have to be performed periodically to adapt classification rules.
  • One approach to overcome these problems, is to develop machine learning algorithms, with which subjects can immediately start using a BCI, without training.
  • A class of machine learning algorithms that might be used to build such a classifier are so-called mixture of experts models [37].

VII. CONCLUSION

  • Signal processing and machine learning methods, as well as applications for BCIs.the authors.
  • One of the main features of this system is that it employs advanced Bayesian machine learning tools which makes training of classifiers simple, fast, and reliable.
  • None of the systems described in the scientific literature is suited for daily use by disabled persons or for use in multimedia environments.
  • This is due to the fact that the technology underlying BCIs is not yet mature enough for usage out of the laboratory.
  • Many challenging and interesting questions in BCI research are thus still waiting to be explored.

Did you find this useful? Give us your feedback

Content maybe subject to copyright    Report

Recent Advances in Brain-Computer Interfaces
(Invited Paper)
Ulrich Hoffmann, Jean-Marc Vesin, Touradj Ebrahimi
Signal Processing Institute
Ecole Polytechnique F´ed´erale de Lausanne (EPFL), Switzerland
Email: {ulrich.hoffmann, jean-marc.vesin, touradj.ebrahimi}@epfl.ch
Abstract A brain-computer interface (BCI) is a communi-
cation system that translates brain activity into commands for
a computer or other devices. In other words, a BCI allows
users to act on their environment by using only brain activity,
without using peripheral nerves and muscles. The major goal of
BCI research is to develop systems that allow disabled users to
communicate with other persons, to control artificial limbs, or
to control their environment. An alternative application area for
brain-computer interfaces (BCIs) lies in the field of multimedia
communication. To develop systems for usage in the field of
assistive technology or multimedia communication, many aspects
of BCI systems are currently being investigated. Research areas
include evaluation of invasive and noninvasive technologies to
measure brain activity, evaluation of control signals (i.e. patterns
of brain activity that can be used for communication), develop-
ment of algorithms for translation of brain signals into computer
commands, and the development of new BCI applications. In this
paper we give an introduction to some of the aspects of BCI
research mentioned above, present a concrete example of a BCI
system, and highlight recent developments and open problems.
I. INTRODUCTION
The ability to communicate with other persons, be it through
speech, gesturing, or writing, is one of the main factors making
the life of any human being enjoyable. Communication is at
the basis of human development, makes it possible to express
ideas, desires, and feelings, and on a more ordinary level
simply allows to cope with daily life.
Individuals suffering from the so-called locked-in syndrome
do not have the above mentioned communication possibilities.
The locked-in syndrome is a condition in which patients
are fully conscious and aware of what is happening in their
environment but are not able to communicate or move. In
fact, the locked-in syndrome is caused by a nearly total
loss of control over the voluntary muscles. A disease that is
known to lead to the locked-in syndrome is amyotrophic lateral
sclerosis (ALS), also known as Lou Gehrig’s disease. ALS is
a progressive, neurodegenerative disease and is characterized
by the death of motor neurons which in turn leads to the loss
of control over voluntary muscles. Besides ALS also multiple
sclerosis, stroke or other cerebrovascular incidents leading to
the infarction or degeneration of parts of the brain can cause
the locked-in syndrome. Clearly, the quality of live of persons
affected by the locked-in syndrome is strongly diminished by
the lack of possibilities to communicate with other persons
and by the complete loss of autonomy.
A promising means to give back basic communication
abilities and a small degree of autonomy to locked-in persons
are BCIs. The idea underlying BCIs is to measure electric,
magnetic, or other physical manifestations of brain activity
and to translate these into commands for a computer or other
devices (see Fig. 1).
From a different angle, BCIs can also be seen as a new and
exciting means of communication that could be used as well
by persons not suffering from disabilities. For example, in the
field of multimedia communication and human-computer inter-
action, BCIs could possibly be used as an additional modality,
together with more traditional modalities, such as the auditive
and visual modalities [1]. Multimodal communication with
the help of a BCI would help to increase the communication
bandwidth between man and machine. Beyond communica-
tion, other applications of BCI involving multimedia can also
be envisioned. For example one can imagine (multiplayer)
games in which BCIs are used for control. Another interesting
application area might be the visualization, or sonification,
i.e. the transformation into sound, of brain activity.
Independently of the application in the fields of assistive
technology or multimedia, the aim of this paper is to give an
introduction to the field of BCI research. In the first part of the
paper (Sections II, III, IV) we review neurophysiologic signals
that can be used in BCIs, signal processing and machine
learning methods for BCIs, and applications for BCIs. In the
second part of the paper (Section V) a concrete state-of-the-art
BCI system is briefly described. Finally, in the third part of
the paper (Section VI) some open problems in BCI research
are mentioned.
Note that the material presented here is strongly biased
towards BCI systems using the electroencephalogram (EEG)
as measurement technology. Other reviews can be found in
[2], [3] and [4].
II. N
EUROPHYSIOLOGIC SIGNALS
To control a BCI, users have to acquire conscious con-
trol over their brain activity. Two fundamentally different
approaches exist to achieve this.
In the first approach, subjects perceive a set of stimuli dis-
played by the BCI system and can control their brain activity
by focusing onto one specific stimulus. The changes in neuro-
physiologic signals resulting from perception and processing
of stimuli are termed event-related potentials (ERPs) and are
discussed together with the corresponding BCI paradigms in
subsection II-A.

Signal Acquisition
Signal Processing
Classification
Application
Feedback
Fig. 1. Building blocks of a BCI. A subject performs a specific cognitive task or concentrates on a specific stimulus. Brain signals are acquired and then
processed with signal processing and classification algorithms. The outcome of the classification is fed into an application, for example a spelling device. The
application generates feedback to inform the subject about the outcome of classification.
In the second approach, users control their brain activity
by concentrating on a specific mental task. For example
imagination of hand movement can be used to modify activity
in the motor cortex. In this approach feedback signals are often
used to let subjects learn the production of easily detectable
patterns of neurophysiologic signals. The types of signals
resulting from concentration on mental tasks together with the
corresponding BCI paradigms are described in subsections II-
B, II-C, and II-D.
A. Event-Related Potentials
ERPs are stereotyped, spatio-temporal patterns of brain
activity, occurring time-locked to an event, for example after
presentation of a stimulus, before execution of a movement,
or after the detection of a novel stimulus.
An example for an ERP that is often used in BCIs is the
so-called P300. The P300 is a positive deflection in the EEG,
appearing approximately 300 ms after the presentation of rare
or surprising, task-relevant stimuli [5]. To evoke the P300,
subjects are asked to observe a random sequence of two types
of stimuli. One stimulus type (the oddball or target stimulus)
appears only rarely in the sequence, while the other stimulus
type (the normal or nontarget stimulus) appears more often.
Whenever the target stimulus appears, a P300 can be observed
in the EEG. This principle was exploited by Farwell and
Donchin in a BCI system which allowed to spell words by
sequentially selecting symbols from a matrix of symbols [6].
Other examples for ERPs that can be used in BCIs are
steady-state visual evoked potentialss (SSVEPs) and motor-
related potentials (MRPs). Descriptions of systems using such
signals can be found in [7] and [8].
B. Oscillatory Brain Activity
Sinusoid like oscillatory brain activity occurs in many
regions of the brain and changes its characteristics according
to the state of subjects, for example between wake and sleep
or between concentrated work and idling. Oscillatory activity
in the EEG is classified into different frequency bands or
rhythms. Typically observable are the delta (1 - 4 Hz), theta
(4 - 8 Hz), alpha and mu (8 - 13 Hz)
1
, beta (13 - 25 Hz), and
gamma (25 - 40 Hz) rhythms.
Among the above mentioned EEG rhythms, especially the
mu-rhythm is of interest because mu-oscillations are decreased
in amplitude when movements of body parts are imagined or
performed. The changes in the mu-rhythm are localized over
the part of the sensorimotor cortex corresponding to the body
part, and so imagined movements of different body parts can
be discriminated. Since the changes in mu- rhythm occurring
in untrained users are usually not strong enough to be detected
by a classification algorithm, feedback training has to be used.
BCI systems employing imagined movements of hands,
feet, or tongue have been mainly introduced by the research
group of Pfurtscheller in Austria [9]. The group of Wolpaw
in the United States has also worked on such systems, and an
impressive sensorimotor rhythm BCI allowing for fast control
of a 2D cursor has been described in [10].
C. Slow Cortical Potentials
Slow cortical potentials (SCPs) are slow voltage shifts in
the EEG occurring in the frequency range 1-2 Hz. Through
feedback training subjects can learn to voluntarily control their
SCPs. The voluntary production of negative and positive SCPs
has been exploited by Birbaumer et al to show that patients
suffering from ALS can use a BCI to control a spelling device
and to communicate with their environment [11].
D. Neuronal Ensemble Activity
Action potentials are thought to be the basic unit of in-
formation in the brain and enable communication between
different neurons. The number of action potentials per time
(the firing rate) can be used in a BCI to predict the behavior of
a subject. For example the firing rate of ensembles of neurons
in the motor and premotor-cortices can be used to predict hand
positions or hand velocities. Neuronal ensemble activity can
thus be employed as neurophysiological signal in BCIs, in
particular in BCIs using microelectrode arrays [12].
1
The term mu-rhythm is used for oscillatory activity with a frequency of
about 10 Hz, localized over the sensorimotor cortex. The term alpha-rhythm
is more general and can be used for any activity in the frequency range 8 -
13 Hz.

III. SIGNAL PROCESSING AND MACHINE LEARNING
In the previous section we have discussed paradigms that
let users control their brain activity and the neurophysiologic
signals corresponding to the respective paradigms. To allow
actual control of a BCI, the neurophysiologic signals have
to be mapped to values that allow to discriminate different
classes of signals, i.e.the neurophysiologic signals have to be
classified. In BCIs, machine learning algorithms are applied
to learn from a training dataset how to classify the signals
of a specific user. As is well known, most machine learning
algorithms can be divided into two modules: signal process-
ing (also known as feature extraction) and classification. In
the following subsections we first review signal processing
methods that are typically used in BCIs and then give a short
introduction to classification methods for BCIs.
A. Time Domain Features
Time domain features are related to changes in the ampli-
tude of neurophysiologic signals, occurring time-locked to the
presentation of stimuli or time-locked to actions of the user of
a BCI. Good examples for signals that can be characterized
with the help of time domain features are the P300 and SCPs.
A strategy that is often used to separate these signals from
background activity and noise is lowpass or bandpass filtering,
optionally followed by downsampling.
An alternative to filtering is to use the wavelet transform of
the signals. Systems based on the discrete wavelet transform
(DWT), as well as systems based on the continuous wavelet
transform (CWT) have been described in the literature.
Besides the use for the EEG signals P300, SCP, and MRP,
time domain features are also used in BCI systems based on
neuronal ensemble activity. A feature that is often used in such
systems is the number of spikes occurring in a certain time
interval. Many different techniques for counting spikes and for
sorting spikes recorded with the same electrode from several
neurons exist. These techniques will however not be further
discussed here.
B. Frequency Domain Features
Frequency domain features are related to changes in oscilla-
tory activity. Since the phase of oscillatory activity is usually
not time-locked to the presentation of stimuli or to actions of
the user, time domain feature extraction techniques cannot be
used. Instead, feature extraction techniques that are invariant
to the exact temporal evolution of signals have to be used.
The most commonly used frequency domain features are
related to changes in the amplitude of oscillatory activity. For
example in systems based on motor imagery, the bandpower
in the mu and beta frequency bands at electrodes located over
the sensorimotor cortex can be used as a feature. To estimate
band power, different methods have been used. These include
Welch’s method [7], adaptive autoregressive models [13], and
Morlet wavelets [14].
A second type of frequency domain features is related to the
synchronization between signals from different brain regions.
Synchronization of signals from different brain regions might
indicate that these regions communicate. This permits to
discriminate cognitive tasks involving communication between
different brain regions [15].
C. Spatial Domain Features
The feature extraction techniques described so far all work
with univariate time series, i.e. data from only one electrode
is used (an exception are synchronization features, extracted
from bivariate time series). In many systems however, data
from more than one electrode is available. Hence, the features
extracted from several electrodes have to be combined in an
efficient way. Finding efficient combinations of features from
more than one electrode is the goal of spatial feature extraction
methods.
The probably simplest way for performing spatial feature
extraction is to use only electrodes that carry useful informa-
tion for discrimination of a given set of cognitive tasks. The
reasoning behind such an approach is that changes in band
power, P300 peaks, or other features do not occur uniformly at
all electrodes but are usually stronger at electrodes over brain
regions implied in the respective cognitive task. Electrodes
can be selected manually or by using an algorithm that
automatically selects an optimal electrode subset.
A spatial feature extraction method that can be used in
addition to electrode selection, consists in applying spatial
filtering algorithms before further processing takes place.
Spatial filtering corresponds to building linear combinations
of the signals measured at several electrodes. Denoting by
s(t) R
E
the signal from E electrodes at time t, spatial
filtering can be expressed as follows:
ˆs(t)=Cs(t). (1)
Here the F ×E matrix C contains the coefficients for F spatial
filters and the vector
ˆ
s(t) R
F
contains the spatially filtered
signals at time t.
To determine the filter coefficients different methods can
be used. For example for motor imagery based BCIs, it
has been shown that spatial filtering with a Laplacian filter
can increase performance [16]. In other methods for spatial
feature extraction, filter coefficients are computed from a set
of training data. An algorithm which is very popular in the
area of motor imagery based BCI systems is the common
spatial patterns (CSP) algorithm [17]. Another method for
computing the coefficients of spatial filters from training data
is independent component analysis (ICA). The assumption
underlying the application of ICA to EEG signals is that the
signals measured on the scalp are a linear and instantaneous
mixture of signals from independent sources in the cortex,
deeper brain structures, and noise [18]. ICA has been mainly
used in P300-based BCIs as a feature extraction method (see
for example [19]).
D. Machine Learning
After feature extraction with one of the methods mentioned
above (or with a combination of methods), supervised machine

learning algorithms are applied to learn how to classify the
signals of a specific user.
A simple but efficient method for supervised machine
learning, appropriate for use in BCIs, is Fisher’s discriminant
analysis (FDA) (see for example [20]). The main advantages
of FDA are that it is a computationally and conceptually
simple method and that very good classification accuracy can
be achieved. A possible drawback of FDA is that a squared-
error loss function is used which makes the method vulnerable
to outliers in the training data. Furthermore, a precondition for
using FDA is that the number of training examples is higher
than the number of dimensions of the training data. In BCI
applications it can happen that this precondition is not fulfilled.
A remedy to the aforementioned problems is to use regularized
FDA [21]. A Bayesian, regularized version of FDA is briefly
described in Section V-C of this paper.
Another algorithm that is relatively often used in BCIs is the
support vector machine (SVM) [20]. The main advantages of
the SVM are that it allows to achieve very good classification
accuracy and that nonlinear classification functions can be eas-
ily implemented by using kernels. A drawback is however, that
training SVMs is computationally complex because regulariza-
tion constants and kernel parameters are typically estimated
with a cross-validation procedure. A second issue is that the
loss function used in the SVM is designed for problems in
which only binary yes/no outputs are needed. The problem
with binary yes/no outputs is that no information is given about
the confidence the system has in those outputs. Confidence
information, or alternatively probabilistic classifier outputs, are
however important to build real-world BCI applications.
Besides FDA and SVM many other machine learning al-
gorithms have been tested in the context of BCI systems. An
overview of these algorithms can be found in [22].
IV. A
PPLICATIONS
In this section we give examples of applications that can be
controlled with a BCI.
A. Spelling Devices
Spelling devices allow severely disabled users to communi-
cate with their environment by sequentially selecting symbols
from the alphabet. One of the first spelling devices mentioned
in the BCI literature is the P300 speller [6]. Another system,
tested with users suffering from ALS and based on SCPs was
described by Birbaumer [11].
B. Environment Control
Environment control systems allow to control electrical ap-
pliances with a BCI. A proof-of-concept environment control
system based on SSVEPs is described in [23]. The control of
a virtual apartment with a BCI using the P300 is described in
[24].
C. Wheelchair Control
A BCI can potentially be used to steer a wheelchair.
Because steering a wheelchair is a complex task and because
wheelchair control has to be extremely reliable, the possible
movements of the wheelchair are strongly constrained in cur-
rent prototype systems. For example in the system presented
in [25], the wheelchair is constrained to move along paths
predefined in software, joining registered locations, and a
P300-based interface is used to select the desired location.
D. Neuromotor Prostheses
The idea underlying research on neuromotor prostheses is
to use a BCI for controlling movement of limbs and to restore
motor function in tetraplegics or amputees. Different types
of neuromotor prostheses can be envisioned depending on the
information transfer rate a BCI provides. If neuronal ensemble
activity is used as control signal, high information transfer
rates are achieved and 3D robotic arms can be controlled [26].
If an EEG based BCI is used, only simple control tasks can
be accomplished. For example in the system described by [27]
sensorimotor rhythms were used to control functional electric
stimulation of hand muscles and so to restore grasp function
in a tetraplegic patient.
E. Gaming and Virtual Reality
Besides the applications targeted towards disabled subjects,
prototypes of gaming and virtual reality applications have been
described in the literature. Examples for such applications are
the control of a spaceship with oscillatory brain activity [28]
and the control of an animated character in an immersive 3D
gaming environment with SSVEPs [7].
V. A
N EFFICIENT BRAIN-COMPUTER INTERFACE FOR
DISABLED SUBJECTS
After the general review of neurophysiologic signals, signal
processing and machine learning methods, and BCI applica-
tions, we now turn our attention to a more detailed description
of a state-of-the-art BCI system. More specifically, a BCI
system using the P300, developed in the Multimedia Signal
Processing Group at the EPFL, is described. The interested
reader can find more details about this system in [29] and
[30].
A. System Description
In the BCI system developed at the EPFL users are facing a
laptop screen on which six images are displayed (see Fig. 2).
The images show a television, a telephone, a lamp, a door,
a window, and a radio. The images are selected according to
an application scenario in which users can control electrical
appliances via a BCI system. The application scenario serves
however only as an example and is not pursued in further
detail.
The images are flashed in random sequences, one image at
a time. Each flash of an image lasts for 100 ms and during
the following 300 ms none of the images is flashed, i.e. the
interstimulus interval (ISI) is 400 ms. The EEG is recorded
at 2048 Hz sampling rate from thirty-two electrodes placed
at the standard positions of the 10-20 international system.
A Biosemi Active Two amplifier is used for amplification

and analog to digital conversion of the EEG signals. Signal
processing and machine learning algorithms are implemented
with MATLAB. The stimulus display and the online access
to the EEG signals are implemented as dynamic link libraries
(DLLs) in C. The DLLs are accessed from MATLAB via a
MEX interface.
B. Materials and Methods
The system is tested with five disabled and four able-bodied
subjects. The disabled subjects suffer from different diseases
such as multiple sclerosis or ALS and are all wheelchair-bound
but have varying communication and limb muscle control
abilities (see [29], [30] for more details). The able-bodied
subjects are PhD students recruited from our laboratory. None
of the able-bodied subjects has known neurological deficits.
Each subject completes four recording sessions. The first
two sessions are performed on one day and the last two
sessions on another day. For all subjects the time between
the first and the last session is less than two weeks. Each of
the sessions consists of six runs, one run for each of the six
images. The following protocol is used in each of the runs.
1) Subjects are asked to count silently how often a pre-
scribed image is flashed (For example: ”Now please
count how often the image with the television is
flashed”).
2) The six images are displayed on the screen and a
warning tone is issued.
3) Four seconds after the warning tone, a random sequence
of flashes is started and the EEG is recorded. The
sequence of flashes is block-randomized, this means
that after six flashes each image is flashed once, after
twelve flashes each image is flashed twice, etc.. The
number of blocks is chosen randomly between 20 and
25. On average 22.5 blocks of six flashes are displayed
in one run, i.e. one run consists on average of 22.5 target
(P300) trials and 22.5 · 5 = 112.5 nontarget (non P300)
trials.
4) In the second, third, and fourth session the target image
Fig. 2. The display used for evoking the P300. Images were flashed, one at
a time, by changing the overall brightness of images.
is inferred from the EEG with a simple classifier
2
. At the
end of each run the image inferred by the classification
algorithm is flashed five times to give feedback to the
user.
5) After each run subjects are asked what their counting
result is. This is done in order to monitor performance
of the subjects.
The duration of one run is approximately one minute and
the duration of one session including setup of electrodes and
short breaks between runs is approximately 30 minutes. One
session comprises on average 810 trials, and the whole data
for one subject consists on average of 3240 trials.
C. Signal Processing and Machine Learning
To extract features from the raw EEG signals, the following
operations are applied:
1) Referencing
The average signal from the two mastoid electrodes is
used for referencing.
2) Filtering
A 6th order forward-backward Butterworth bandpass
filter is used to filter the data. Cutoff frequencies are set
to 1.0 Hz and 12.0 Hz. The MATLAB function butter is
used to compute the filter coefficients and the function
filtfilt is used for filtering.
3) Downsampling
The EEG is downsampled from 2048 Hz to 32 Hz by
selecting each 64th sample from the bandpass-filtered
data.
4) Single Trial Extraction
Single trials of duration 1000 ms are extracted from
the data. Single trials start at stimulus onset, i.e. at the
beginning of the intensification of an image, and end
1000 ms after stimulus onset. Due to the ISI of 400 ms,
the last 600 ms of each trial overlap with the first 600
ms of the following trial.
5) Windsorizing
Eye blinks, eye movement, muscle activity, or subject
movement can cause large amplitude outliers in the
EEG. To reduce the effects of such outliers, the data
from each electrode are windsorized. For the samples
from each electrode the 10th percentile and the 90th
percentile are computed. Amplitude values lying below
the 10th percentile or above the 90th percentile are then
replaced by the 10th percentile or the 90th percentile,
respectively.
6) Scaling
The samples from each electrode are scaled to the
interval [1, 1].
7) Electrode Selection
For the results presented in this paper data from a set of
eight electrodes is used. The electrode set consists of the
2
The classifier is trained from the data recorded in the first session. The
algorithm described in [31] is used for preprocessing and the algorithm
described in [32] is used for classification.

Citations
More filters
Proceedings ArticleDOI
16 Apr 2013
TL;DR: The main objective of this paper is to predict preterm deliveries at an early gestation period using uterine electromyography signals (EMG), and two ways will be taken to take some uterine EMG linear parameters as features to a suitable neural network.
Abstract: The main objective of this paper is to predict preterm deliveries at an early gestation period using uterine electromyography signals (EMG). Detecting such uterine signals can yield a promising approach to detennine and take actions to prevent this potential risk. Previous classification studies use only linear methods as classic spectral analysis to classify the uterine EMG that does not give clinically useful results. On another hand some studies make linear and non-linear analysis for the uterine EMG and find that the non-linear parameters can distinguish the preterm delivery uterine EMG from the term one. In this research, two ways will be taken combining the two previousideas;the first way is to take some uterine EMG linear parameters as features to a suitable neural network and the second one is to take some uterine EMG non-linear parameters as features to the same neural network. Then, the two ways' results are compared using ROC analysis which provesthat the chance of correctly classification increases markedly when applying the non-linear methods.

14 citations

Journal Article
TL;DR: A combination of PCA with Linear Discriminant Analysis and neural networks provided as high as 13\% accuracy gain for single-trial classification of P300 response while using only 3 to 4 principal components.
Abstract: The P300 event-related potential (ERP), evoked in scalp-recorded electroencephalography (EEG) by external stimuli, has proven to be a reliable response for controlling a BCI. The P300 component of an event related potential is thus widely used in brain-computer interfaces to translate the subjects' intent by mere thoughts into commands to control artificial devices. The main challenge in the classification of P300 trials in electroencephalographic (EEG) data is the low signal-to-noise ratio (SNR) of the P300 response. To overcome the low SNR of individual trials, it is common practice to average together many consecutive trials, which effectively diminishes the random noise. Unfortunately, when more repeated trials are required for applications such as the P300 speller, the communication rate is greatly reduced. This has resulted in a need for better methods to improve single-trial classification accuracy of P300 response. In this work, we use Principal Component Analysis (PCA) as a preprocessing method and use Linear Discriminant Analysis (LDA)and neural networks for classification. The results show that a combination of PCA with these methods provided as high as 13\% accuracy gain for single-trial classification while using only 3 to 4 principal components.

14 citations


Cites background or methods from "Recent advances in brain-computer i..."

  • ...In the first method, the external stimuli cause changes in neurophysiologic signals called event-related potentials (ERPs) [15, 22] which are used to identify a user’s response to the stimuli presented....

    [...]

  • ...The idea underlying BCIs is to measure electric, magnetic, or other physical manifestations of the brain activity and to translate these into commands for a computer or other devices [21, 15]....

    [...]

  • ...For example, imagination of hand movement can be used to modify activity in the motor cortex [15]....

    [...]

Proceedings ArticleDOI
13 May 2010
TL;DR: The present work is oriented to offer a communication system to people who undergo a severe loss of motor function as a result of various accidents and/or diseases so that they can control and interact better with the environment through the acquisition of EEG signals by electrodes and implementation of algorithms to extract characteristics.
Abstract: The present work is oriented to offer a communication system to people who undergo a severe loss of motor function as a result of various accidents and/or diseases so that they can control and interact better with the environment, for which a brain-computer interface has been implemented through the acquisition of EEG signals by electrodes and implementation of algorithms to extract characteristics and execute a method of classification that would interpret these signals and execute corresponding actions The first objective is to design and construct a system of communication and control based on the thought, able to catch and measure EEG signals. The second objective is to implement the system of data acquisition including a digital filter in real time that allows us to eliminate the noise. The third objective is to analyze the variation of the EEG signals in front of the different tasks under study and of implementing an algorithm of extraction of characteristics. The fourth objective is to work on the basis of the characteristics of the EEG signals, to implement a classification system that can discriminate between the two tasks under study on the basis of the corresponding battles.

12 citations

Book ChapterDOI
29 Apr 2012
TL;DR: This paper analyzes shrinkage functions for signal filtering and proposes a class-adaptive method for EEG data denoising and the results are evaluated using a Support Vector Machine.
Abstract: Brain-computer interface (BCI) systems use electro-encephalogram (EEG) data to control external electronic devices. The main task of BCI systems is to differentiate the classes of mental tasks from the EEG data. The EEG data is inherently complex and difficult to analyze due to interference by eye and muscle movements as well as electrical grid noise. In this paper we analyze shrinkage functions for signal filtering and propose a class-adaptive method for EEG data denoising. The results are evaluated using a Support Vector Machine.

12 citations

Journal ArticleDOI
TL;DR: A novel therapy to recover patients from drug craving diseases, with the use of brain–computer interfaces (BCIs), and the Naive Bayes method has been chosen as the best classifier between the tested ones, giving a +12.21% performance boost as concerns the F1-score metric.

11 citations

References
More filters
Book
28 Jul 2013
TL;DR: In this paper, the authors describe the important ideas in these areas in a common conceptual framework, and the emphasis is on concepts rather than mathematics, with a liberal use of color graphics.
Abstract: During the past decade there has been an explosion in computation and information technology. With it have come vast amounts of data in a variety of fields such as medicine, biology, finance, and marketing. The challenge of understanding these data has led to the development of new tools in the field of statistics, and spawned new areas such as data mining, machine learning, and bioinformatics. Many of these tools have common underpinnings but are often expressed with different terminology. This book describes the important ideas in these areas in a common conceptual framework. While the approach is statistical, the emphasis is on concepts rather than mathematics. Many examples are given, with a liberal use of color graphics. It is a valuable resource for statisticians and anyone interested in data mining in science or industry. The book's coverage is broad, from supervised learning (prediction) to unsupervised learning. The many topics include neural networks, support vector machines, classification trees and boosting---the first comprehensive treatment of this topic in any book. This major new edition features many topics not covered in the original, including graphical models, random forests, ensemble methods, least angle regression and path algorithms for the lasso, non-negative matrix factorization, and spectral clustering. There is also a chapter on methods for ``wide'' data (p bigger than n), including multiple testing and false discovery rates. Trevor Hastie, Robert Tibshirani, and Jerome Friedman are professors of statistics at Stanford University. They are prominent researchers in this area: Hastie and Tibshirani developed generalized additive models and wrote a popular book of that title. Hastie co-developed much of the statistical modeling software and environment in R/S-PLUS and invented principal curves and surfaces. Tibshirani proposed the lasso and is co-author of the very successful An Introduction to the Bootstrap. Friedman is the co-inventor of many data-mining tools including CART, MARS, projection pursuit and gradient boosting.

19,261 citations

Journal ArticleDOI
TL;DR: This book covers a broad range of topics for regular factorial designs and presents all of the material in very mathematical fashion and will surely become an invaluable resource for researchers and graduate students doing research in the design of factorial experiments.
Abstract: (2007). Pattern Recognition and Machine Learning. Technometrics: Vol. 49, No. 3, pp. 366-366.

18,802 citations

Journal ArticleDOI
TL;DR: With adequate recognition and effective engagement of all issues, BCI systems could eventually provide an important new communication and control option for those with motor disabilities and might also give those without disabilities a supplementary control channel or a control channel useful in special circumstances.

6,803 citations


"Recent advances in brain-computer i..." refers background or methods in this paper

  • ...The maximal bitrate, computed according to the definition of Wolpaw [3], was approximately 25 bits/min....

    [...]

  • ...Other reviews can be found in [2], [3] and [4]....

    [...]

  • ...The group of Wolpaw in the United States has also worked on such systems, and an impressive sensorimotor rhythm BCI allowing for fast control of a 2D cursor has been described in [10]....

    [...]

Journal ArticleDOI
01 May 1992
TL;DR: The Bayesian approach to regularization and model-comparison is demonstrated by studying the inference problem of interpolating noisy data by examining the posterior probability distribution of regularizing constants and noise levels.
Abstract: Although Bayesian analysis has been in use since Laplace, the Bayesian method of model-comparison has only recently been developed in depth. In this paper, the Bayesian approach to regularization and model-comparison is demonstrated by studying the inference problem of interpolating noisy data. The concepts and methods described are quite general and can be applied to many other data modeling problems. Regularizing constants are set by examining their posterior probability distribution. Alternative regularizers (priors) and alternative basis sets are objectively compared by evaluating the evidence for them. Occam's razor is automatically embodied by this process. The way in which Bayes infers the values of regularizing constants and noise levels has an elegant interpretation in terms of the effective number of parameters determined by the data set. This framework is due to Gull and Skilling.

4,194 citations


"Recent advances in brain-computer i..." refers background in this paper

  • ...The maximum likelihood solution for the hyperparameters can be found with a simple iterative algorithm which we do not discuss in detail here but which is described in [29], [30], [34]....

    [...]

Journal ArticleDOI
13 Jul 2006-Nature
TL;DR: Initial results for a tetraplegic human using a pilot NMP suggest that NMPs based upon intracortical neuronal ensemble spiking activity could provide a valuable new neurotechnology to restore independence for humans with paralysis.
Abstract: Neuromotor prostheses (NMPs) aim to replace or restore lost motor functions in paralysed humans by routeing movement-related signals from the brain, around damaged parts of the nervous system, to external effectors. To translate preclinical results from intact animals to a clinically useful NMP, movement signals must persist in cortex after spinal cord injury and be engaged by movement intent when sensory inputs and limb movement are long absent. Furthermore, NMPs would require that intention-driven neuronal activity be converted into a control signal that enables useful tasks. Here we show initial results for a tetraplegic human (MN) using a pilot NMP. Neuronal ensemble activity recorded through a 96-microelectrode array implanted in primary motor cortex demonstrated that intended hand motion modulates cortical spiking patterns three years after spinal cord injury. Decoders were created, providing a ‘neural cursor’ with which MN opened simulated e-mail and operated devices such as a television, even while conversing. Furthermore, MN used neural control to open and close a prosthetic hand, and perform rudimentary actions with a multi-jointed robotic arm. These early results suggest that NMPs based upon intracortical neuronal ensemble spiking activity could provide a valuable new neurotechnology to restore independence for humans with paralysis. The cover shows Matt Nagle, first participant in the BrainGate pilot clinical trial. He is unable to move his arms or legs following cervical spinal cord injury. Researchers at the Department of Neuroscience at Brown University, working with biotech company Cyberkinetics and 3 other institutions, demonstrate that movement-related signals can be relayed from the brain via an implanted BrainGate chip, allowing the patient to drive a computer screen cursor and activate simple robotic devices. Such neuromotor prostheses could pave the way towards systems to replace or restore lost motor function in paralysed humans. Prior to this advance, this type of work has been performed chiefly in monkeys. The latest example of such research has achieved a large increase in speed over current devices, enhancing the prospects for clinically viable brain-machine interfaces.

3,120 citations


"Recent advances in brain-computer i..." refers methods in this paper

  • ...Neuronal ensemble activity can thus be employed as neurophysiological signal in BCIs, in particular in BCIs using microelectrode arrays [12]....

    [...]

Frequently Asked Questions (19)
Q1. What are the contributions mentioned in the paper "Recent advances in brain-computer interfaces" ?

In this paper the authors give an introduction to some of the aspects of BCI research mentioned above, present a concrete example of a BCI system, and highlight recent developments and open problems. 

The main advantages of the SVM are that it allows to achieve very good classification accuracy and that nonlinear classification functions can be easily implemented by using kernels. 

A strategy that is often used to separate these signals from background activity and noise is lowpass or bandpass filtering, optionally followed by downsampling. 

The EEG is recorded at 2048 Hz sampling rate from thirty-two electrodes placed at the standard positions of the 10-20 international system. 

Other examples for ERPs that can be used in BCIs are steady-state visual evoked potentialss (SSVEPs) and motorrelated potentials (MRPs). 

The assumption underlying the application of ICA to EEG signals is that the signals measured on the scalp are a linear and instantaneous mixture of signals from independent sources in the cortex, deeper brain structures, and noise [18]. 

A drawback is however, that training SVMs is computationally complex because regularization constants and kernel parameters are typically estimated with a cross-validation procedure. 

The main advantages of FDA are that it is a computationally and conceptually simple method and that very good classification accuracy can be achieved. 

Besides the use for the EEG signals P300, SCP, and MRP, time domain features are also used in BCI systems based on neuronal ensemble activity. 

Through a Bayesian analysis the degree of regularization can be estimated automatically and quickly from training data without the need for time consuming cross-validation. 

The idea underlying research on neuromotor prostheses is to use a BCI for controlling movement of limbs and to restore motor function in tetraplegics or amputees. 

One approach to overcome these problems, is to develop machine learning algorithms, with which subjects can immediately start using a BCI, without training. 

Neuronal ensemble activity can thus be employed as neurophysiological signal in BCIs, in particular in BCIs using microelectrode arrays [12]. 

Because steering a wheelchair is a complex task and becausewheelchair control has to be extremely reliable, the possible movements of the wheelchair are strongly constrained in current prototype systems. 

A second issue is that the loss function used in the SVM is designed for problems in which only binary yes/no outputs are needed. 

While due to fatigue or concentration problems not all ablebodied subjects achieved 100% classification accuracy, the bitrates for the able-bodied subjects were in general higher than those of the disabled subjects. 

The types of signals resulting from concentration on mental tasks together with the corresponding BCI paradigms are described in subsections IIB, II-C, and II-D.ERPs are stereotyped, spatio-temporal patterns of brain activity, occurring time-locked to an event, for example after presentation of a stimulus, before execution of a movement, or after the detection of a novel stimulus. 

algorithms that can detect if the user wants to communicate via the BCI or is engaged in other activity have to be developed. 

For three of the other four disabled subjects tested in their study 100% classification accuracy was also achieved and the maximal bitrate varied between 9 and 19 bits/min.