A brain-actuated wheelchair: asynchronous and non-invasive Brain-computer interfaces for continuous control of robots.

TL;DR: The results show that subjects can rapidly master the authors' asynchronous EEG-based BCI to control a wheelchair and can autonomously operate the BCI over long periods of time without the need for adaptive algorithms externally tuned by a human operator to minimize the impact of EEG non-stationarities.

About: This article is published in Clinical Neurophysiology.The article was published on 2008-09-01 and is currently open access. It has received 644 citations till now. The article focuses on the topics: Wheelchair.

Summary

1. Introduction

• IDIAP Research Institute, Centre du Parc., Av. des Prés-Beudin 20, CH-1920 Martigny, Switzerland, also known as Address.
• Secondly, adaptation in the wrong moment (e.g., when the user is not properly executing the mental tasks because of fatigue, distraction, etc) will incorrectly change the feedback (the device’s behavior) and will disrupt user’s learning process.

2.1. EEG data acquisition and preprocessing

• EEG Data were recorded from two healthy subjects with a portable Biosemi acquisition system using 64 channels sampled at 512 Hz and high-pass filtered at 1 Hz.
• Then, the signal was spatially filtered using a common average reference (CAR) before estimating the power spectral density (PSD) in the band 8–48 Hz with 2-Hz resolution over the last 1 s. The PSD was estimated every 62.5 ms (i.e., 16 times per second) using the Welch method with five overlapped (25%) Hanning windows of 500 ms.
• Obviously, not all these 1344 features are used as control signals.
• Sections 2.2 and 2.3 describe the algorithms to estimate the relevance of the features for discriminating the mental commands and the procedure to select the most stable discriminant features that will be fed to the classifier embedded in the BCI.
• This classifier processes each of the EEG samples and the BCI combines eight consecutive responses to deliver a mental command every 0.5 s.

2.2. Calibration Sessions and Feature Extraction

• To extract stable discriminant EEG features (see Section 2.3.2) and build the statistical Gaussian classifier embedded in the BCI (see Section 2.3.3), both subjects participated in 20 calibration sessions recorded in the same day as the test driving session 1.
• The subjects were instructed to execute the three mental tasks (left hand imagination movement, rest, and words association),2 tasks utilized as mental commands to operate the wheelchair, in a self-paced way.
• The data from the 20 calibration sessions were grouped in four blocks (B1, B2, B3 and B4) of five consecutive sessions.
• Afterwards, for each selected frequency, the authors took the configuration of electrodes (out of the 15 possible ones) that yielded the highest classification accuracy on the configuration (B1 + B2 + B3) B4.

2.3. System description

• The system is integrated by two entities, the intelligent wheelchair and the BCI system.
• Environmental information from the wheelchair’s sensors feeds a contextual filter that builds a probability distribution PEnv(C) over the possible user’s mental steering commands, C = {Left, Right, Forward}.
• The BCI system estimates the probabilities PEEG(C) of the different mental commands from the EEG signals.
• Both streams of information are combined to produce a filtered estimate of the user’s intent P(C) = PEEG(C) PEnv(C).
• The shared control system also uses the environmental information from the wheelchair’s.

2.3.1. Context-based filter

• Context estimation is done by defining a general, a priori-known user intention (smooth and efficient forward navigation through the environment) on the one hand and a constant automatic estimation of the environmental situation on the other hand.
• Each opening then represents a general direction in which the user might opt to continue his navigation.
• A probability distribution concerning the possible local user actions is built.
• If the wheelchair is oriented 45 North–West, PEnv becomes zero, while the possible commands now are Left and Right, with equal probability, reflecting the belief that one of the orthogonal directions North or West should be chosen.
• See Vanacker et al. (2007) for a detailed description.

2.3.2. Feature extractor

• The authors approach is based on a mutual learning process where the user and the BCI are coupled together and adapt to each other.
• To facilitate and accelerate this process, it is necessary to select the relevant EEG features that best discriminate among the mental tasks executed by the user.
• The feature selection process is based on Canonical Variates Analysis (CVA) (Krzanowski, 1988), also known as Multiple Discriminant Analysis (Duda et al., 2001), which provides a canonical solution for multi-class problems.
• Note that the direction of the eigenvectors A maximizes the quotient between the between-classes dispersion matrix B and the pooled within-classes dispersion matrix W. Thus, the CDSPs are obtained by projecting X = SA.

2.3.3. Classifier

• The classifier utilized is a statistical Gaussian classifier (see Millán et al., 2004 for more details).
• Each class is represented by a number of Gaussian prototypes, typically less than four.
• The classifier output for input vector x is then the class with the highest probability.
• In order to smooth this output, the authors average the class-conditioned probabilities of the last eight consecutive input vectors x.
• During offline training of the classifier, the prototype centers are initialized by any clustering algorithm or generative approach.

2.4.1. Task 1

• The subjects were asked to mentally drive the simulated wheelchair from a starting point to a goal following a pre-specified path by executing three different mental tasks (left hand imagination movement to turn Left, rest to go Forward, and words association to turn Right).
• Fig. 1 depicts the monitor display and the pre-specified path.
• Every subject participated in five experimental sessions, each consisting of 10 trials.
• The time elapsed between two consecutive experimental sessions was variable to assess the system robustness over time: 1 day between sessions 1 and 2, 2 months between sessions 2 and 3, 1 h between sessions 3 and 4, and finally 1 day between sessions 4 and 5.

2.4.2. Task 2

• To further assess the performance of the brain-actuated wheelchair, Subject 1 participated in a second experiment four months later.
• He performed 10 trials in the same simulated environment where he was asked to drive the simulated wheelchair following 10 different complex and random paths never tried before.
• Fig. 6 depicts the 10 complex and random paths.
• Subject 2 did not participate in this task because she was not available.

2.5. Analysis

• The system’s robustness was assessed in task 1 on three criteria, namely the percentage of goals reached, the BCI classification accuracy, and the shared control accuracy (the actual mental commands sent to the wheelchair after combining the probability distributions from the BCI and contextual filter).
• For the contextual analysis, the path was split into seven stretches.
• It follows that using the subject’s stated intent for labelling data yields a pessimistic and/or wrong estimate of the accuracies of the BCI and the shared control.
• Only those samples where the subject’s stated intent corresponds to the stretch label were utilized to compute the accuracies.
• The analysis of EOG and EMG activity showed that eye movements were equally distributed among the classes and that there was no significant muscular activity.

3.1.1. Global performance

• Fig. 8 depicts the percentage of final goals reached over the five sessions for the two experimental subjects.
• Furthermore, shared control helped to generate smoother trajectories, especially in the vicinity of walls.
• Subject 1 always correctly performed the optimal action for all other stretches he went through.
• Finally, in sessions 2 and 4 subject 2 reached the final goal in 70% (or more) of the trials and each local goal in more than 80% of the trials.

3.1.2. System performance in single trials

• Here the authors analyze the performance of the brain-actuated wheelchair in a few single trials to illustrate emergent behaviors originated by the interaction of the BCI system and the shared control system in particular contexts.
• The authors have already mentioned in the previous section that, for some stretches, shared control degraded the performance of the BCI, what could indicate that subjects tried to deliver mental commands that the shared control system considers impossible to execute.
• Subject 1 always succeeded in making the wheelchair turn Right.
• But for the subjects to learn that model they need to have a stable performance of the brain-actuated wheelchair.

3.2. Task 2

• Subject 1 reached the final goal in 80% of the trials.
• Right turn requires a very high BCI performance because the subject has to rotate the wheelchair by 90 being almost in the same place (i.e., without entering the corridor it is facing).
• Indeed, the execution of even a short number of wrong commands in this context makes the shared control system to move the wheelchair Forward.
• Once the wheelchair is in the corridor, the shared control system makes it very hard to turn back (180 ) rapidly and the trial is considered a failure.
• To illustrate the behavior of the brain-actuated wheelchair in this task, the authors have included a video clip (see the Supplementary video) which contains the trajectories generated on trials 7 and 10 (un).

4. Conclusions

• In this paper the authors have presented an asynchronous and non-invasive EEG-based BCI prototype for brain-actuated wheelchair driving.
• First, the selection of stable user-specific EEG features that maximize the separability between the patterns generated by executing different mental tasks.
• This calibration procedure, which is common to all users, selects user-specific features that are relevant and stable.
• Contrarily, the authors follow a mutual learning process to facilitate and accelerate the user’s training period.
• As shown for the experiments in task 1, subjects can control the wheelchair since the first day with a performance significantly better than a random BCI.

Figures

Fig. 6. Top view of the random paths in Task 2. Trial 1 placed in upper row, first column. Trial 10 placed in second row, last column.

Fig. 7. Top view of the world and the path stretches. Stretches F1 and F2 were labelled as Forward, R1 and R2 labelled as Right, L labelled as Left, and SD1 and SD2 labelled as strategy dependent. The subjects can go through SD1 by means of two strategies, either executing Forward or executing Right followed by Left. Through SD2, subjects can execute either Forward or Left followed by Right.

Table 1 LDA train-test classification accuracies on the configuration

Fig. 1. (Left) Monitor display in a first person view from the Start. The white cursor at the center is the fixation point. The rectangle at the bottom is the simulated wheelchair. (Right) Top view of the simulated world and the pre-specified path.

Fig. 8. Percentage of reached final goals over sessions. The time elapsed between sessions was: 1 day between sessions 1 and 2, 2 months between sessions 2 and 3, 1 h between sessions 3 and 4, and 1 day between sessions 4 and 5.

Fig. 2. Electrode discrimination index values De (see Section 2.3.2) for the selected frequencies for each subject, and the associated scalp distribution of the averaged logarithmic transform of the power spectral density, Log(PSDe), for each class. For subject 1, De is higher at left temporal, central and right occipital areas. For subject 2, at 10 Hz it is higher at right centro-parietal areas, and at 12 and 14 Hz it is higher at bilateral parietal areas. These areas correspond with those where the differences between the averaged Log(PSDe) patterns associated to each mental task is the biggest.

Table 2 Percentage of local goals reached (subgoals), average BCI classification accuracy and average shared control accuracy over the 7 path stretches

Table 3 Inter-trial differences in performance: subject 1, session 4

Fig. 4. Principle of context estimator. With a laser range scanner, a set of regions that provide safe manoeuvrable openings in the environment is detected. The figure shows how the region to the left and the one in front of the wheelchair are detected as openings.

Fig. 5. Environmental belief over the four quadrants in function of the wheelchair orientation in Fig. 4. The inner circle shows the probability of a Right command P(R), the middle circle shows the probability of a Left command P(L), and the outer circle the probability of a Forward command P(F). The figure also shows the resulting probabilities of the three steering command along each direction.

Fig. 3. Architecture of the brain-actuated wheelchair.