Showing papers in "Journal of Neural Engineering in 2020"

BCI for stroke rehabilitation: motor and beyond.

Abstract: Stroke is one of the leading causes of long-term disability among adults and contributes to major socio-economic burden globally. Stroke frequently results in multifaceted impairments including motor, cognitive and emotion deficits. In recent years, brain-computer interface (BCI)-based therapy has shown promising results for post-stroke motor rehabilitation. In spite of the success received by BCI-based interventions in the motor domain, non-motor impairments are yet to receive similar attention in research and clinical settings. Some preliminary encouraging results in post-stroke cognitive rehabilitation using BCI seem to suggest that it may also hold potential for treating non-motor deficits such as cognitive and emotion impairments. Moreover, past studies have shown an intricate relationship between motor, cognitive and emotion functions which might influence the overall post-stroke rehabilitation outcome. A number of studies highlight the inability of current treatment protocols to account for the implicit interplay between motor, cognitive and emotion functions. This indicates the necessity to explore an all-inclusive treatment plan targeting the synergistic influence of these standalone interventions. This approach may lead to better overall recovery than treating the individual deficits in isolation. In this paper, we review the recent advances in BCI-based post-stroke motor rehabilitation and highlight the potential for the use of BCI systems beyond the motor domain, in particular, in improving cognition and emotion of stroke patients. Building on the current results and findings of studies in individual domains, we next discuss the possibility of a holistic BCI system for motor, cognitive and affect rehabilitation which may synergistically promote restorative neuroplasticity. Such a system would provide an all-encompassing rehabilitation platform, leading to overarching clinical outcomes and transfer of these outcomes to a better quality of living. This is one of the first works to analyse the possibility of targeting cross-domain influence of post-stroke functional recovery enabled by BCI-based rehabilitation.

HS-CNN: a CNN with hybrid convolution scale for EEG motor imagery classification.

Abstract: Objective Electroencephalography (EEG) motor imagery classification has been widely used in healthcare applications such as mobile assistive robots and post-stroke rehabilitation. Recently, EEG motor imagery classification methods based on convolutional neural networks (CNNs) have been proposed and have achieved relatively high classification accuracy. However, these methods use single convolution scale in the CNN, while the best convolution scale differs from subject to subject. This limits the classification accuracy. Another issue is that the classification accuracy degrades when training data is limited. Approach To address these issues, we have proposed a hybrid-scale CNN architecture with a data augmentation method for EEG motor imagery classification. Main results Compared with several state-of-the-art methods, the proposed method achieves an average classification accuracy of 91.57% and 87.6% on two commonly used datasets, which outperforms several state-of-the-art EEG motor imagery classification methods. Significance The proposed method effectively addresses the issues of existing CNN-based EEG motor imagery classification methods and improves the classification accuracy.

Review of semi-dry electrodes for EEG recording

Abstract: Developing reliable and user-friendly electroencephalography (EEG) electrodes remains a challenge for emerging real-world EEG applications. Classic wet electrodes are the gold standard for recording EEG; however, they are difficult to implement and make users uncomfortable, thus severely restricting their widespread application in real-life scenarios. An alternative is dry electrodes, which do not require conductive gels or skin preparation. Despite their quick setup and improved user-friendliness, dry electrodes still have some inherent problems (invasive, relatively poor signal quality, or sensitivity to motion artifacts), which limit their practical utilization. In recent years, semi-dry electrodes, which require only a small amount of electrolyte fluid, have been successfully developed, combining the advantages of both wet and dry electrodes while addressing their respective drawbacks. Semi-dry electrodes can collect reliable EEG signals comparable to wet electrodes. Moreover, their setup is as fast and convenient similar to that of dry electrodes. Hence, semi-dry electrodes have shown tremendous application prospects for real-world EEG acquisition. Herein, we systematically summarize the development, evaluation methods, and practical design considerations of semi-dry electrodes. Some feasible suggestions and new ideas for the development of semi-dry electrodes have been presented. This review provides valuable technical support for the development of semi-dry electrodes toward emerging practical applications.

Enhance decoding of pre-movement EEG patterns for brain-computer interfaces.

Abstract: OBJECTIVE In recent years, brain-computer interface (BCI) systems based on electroencephalography (EEG) have developed rapidly. However, the decoding of voluntary finger pre-movements from EEG is still a challenge for BCIs. This study aimed to analyze the pre-movement EEG features in time and frequency domains and design an efficient method to decode the movement-related patterns. APPROACH In this study, we first investigated the EEG features induced by the intention of left and right finger movements. Specifically, the movement-related cortical potential (MRCP) and event-related desynchronization (ERD) features were extracted using discriminative canonical pattern matching (DCPM) and common spatial patterns (CSP), respectively. Then, the two types of features were classified by two fisher discriminant analysis (FDA) classifiers, respectively. Their decision values were further assembled to facilitate the classification. To verify the validity of the proposed method, a private dataset containing 12 subjects and a public dataset from BCI competition II were used for estimating the classification accuracy. MAIN RESULTS As a result, for the private dataset, the combination of DCPM and CSP achieved an average accuracy of 80.96%, which was 5.08% higher than the single DCPM method (p < 0.01) and 10.23% higher than the single CSP method (p < 0.01). Notably, the highest accuracy could achieve 91.5% for the combination method. The test accuracy of dataset IV of BCI competition II was 90%, which was equal to the best result in the existing literature. SIGNIFICANCE The results demonstrate the MRCP and ERD features of pre-movements contain significantly discriminative information, which are complementary to each other, and thereby could be well recognized by the proposed combination method of DCPM and CSP. Therefore, this study provides a promising approach for the decoding of pre-movement EEG patterns, which is significant for the development of BCIs.

A novel approach of decoding EEG four-class motor imagery tasks via scout ESI and CNN.

Abstract: Objective. To develop and implement a novel approach which combines the technique of scout EEG source imaging (ESI) with convolutional neural network (CNN) for the classification of motor imagery (MI) tasks. Approach. The technique of ESI uses a boundary element method (BEM) and weighted minimum norm estimation (WMNE) to solve the EEG forward and inverse problems, respectively. Ten scouts are then created within the motor cortex to select the region of interest (ROI). We extract features from the time series of scouts using a Morlet wavelet approach. Lastly, CNN is employed for classifying MI tasks. Main results. The overall mean accuracy on the Physionet database reaches 94.5% and the individual accuracy of each task reaches 95.3%, 93.3%, 93.6%, 96% for the left fist, right fist, both fists and both feet, correspondingly, validated using ten-fold cross validation. We report an increase of up to 14.4% for overall classification compared with the competitive results from the state-of-the-art MI classification methods. Then, we add four new subjects to verify the validity of the method and the overall mean accuracy is 92.5%. Furthermore, the global classifier was adapted to single subjects improving the overall mean accuracy to 94.54%. Significance. The combination of scout ESI and CNN enhances BCI performance of decoding EEG four-class MI tasks.

Common misconceptions, hidden biases and modern challenges of dMRI tractography.

Abstract: The human brain is a complex and organized network, where the connection between regions is not achieved with single axons crisscrossing each other but rather millions of densely packed and well-ordered axons. Reconstruction from diffusion MRI tractography is only an attempt to capture the full complexity of this network, at the macroscale. This review provides an overview of the misconceptions, biases and pitfalls present in structural white matter bundle and connectome reconstruction using tractography. The goal is not to discourage readers, but rather to inform them of the limitations present in the methods used by researchers in the field in order to focus on what they can do and promote proper interpretations of their results. It also provides a list of open problems that could be solved in future research projects for the next generation of PhD students.

A practical guide to methodological considerations in the controllability of structural brain networks.

Abstract: Objective: Predicting how the brain can be driven to specific states by means of internal or external control requires a fundamental understanding of the relationship between neural connectivity and activity. Network control theory is a powerful tool from the physical and engineering sciences that can provide insights regarding that relationship; it formalizes the study of how the dynamics of a complex system can arise from its underlying structure of interconnected units. Approach: Given the recent use of network control theory in neuroscience, it is now timely to offer a practical guide to methodological considerations in the controllability of structural brain networks. Here we provide a systematic overview of the framework, examine the impact of modeling choices on frequently studied control metrics, and suggest potentially useful theoretical extensions. We ground our discussions, numerical demonstrations, and theoretical advances in a dataset of high-resolution diffusion imaging with 730 diffusion directions acquired over approximately 1 hour of scanning from ten healthy young adults. Main results: Following a didactic introduction of the theory, we probe how a selection of modeling choices affects four common statistics: average controllability, modal controllability, minimum control energy, and optimal control energy. Next, we extend the current state-of-the-art two ways: first, by developing an alternative measure of structural connectivity that accounts for radial propagation of activity through abutting tissue, and second, by defining a complementary metric quantifying the complexity of the energy landscape of a system. We close with specific modeling recommendations and a discussion of methodological constraints. Significance: Our hope is that this accessible account will inspire the neuroimaging community to more fully exploit the potential of network control theory in tackling pressing questions in cognitive, developmental, and clinical neuroscience.

Data augmentation for enhancing EEG-based emotion recognition with deep generative models

Abstract: Objective The data scarcity problem in emotion recognition from electroencephalography (EEG) leads to difficulty in building an affective model with high accuracy using machine learning algorithms or deep neural networks. Inspired by emerging deep generative models, we propose three methods for augmenting EEG training data to enhance the performance of emotion recognition models. Approach Our proposed methods are based on two deep generative models, variational autoencoder (VAE) and generative adversarial network (GAN), and two data augmentation ways, full and partial usage strategies. For the full usage strategy, all of the generated data are augmented to the training dataset without judging the quality of the generated data, while for the partial usage, only high-quality data are selected and appended to the training dataset. These three methods are called conditional Wasserstein GAN (cWGAN), selective VAE (sVAE), and selective WGAN (sWGAN). Main results To evaluate the effectiveness of these proposed methods, we perform a systematic experimental study on two public EEG datasets for emotion recognition, namely, SEED and DEAP. We first generate realistic-like EEG training data in two forms: power spectral density and differential entropy. Then, we augment the original training datasets with a different number of generated realistic-like EEG data. Finally, we train support vector machines and deep neural networks with shortcut layers to build affective models using the original and augmented training datasets. The experimental results demonstrate that our proposed data augmentation methods based on generative models outperform the existing data augmentation approaches such as conditional VAE, Gaussian noise, and rotational data augmentation. We also observe that the number of generated data should be less than 10 times of the original training dataset to achieve the best performance. Significance The augmented training datasets produced by our proposed sWGAN method significantly enhance the performance of EEG-based emotion recognition models.

Learning across multi-stimulus enhances target recognition methods in SSVEP-based BCIs

Abstract: alt;iagt;Objectivealt;/iagt;: Latest target recognition methods that are equipped with learning from subject's calibration data, represented by the extended canonical correlation analysis (eCCA) and the ensemble task-related component analysis (eTRCA), can achieve extra high performance in the steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs), however their performance deteriorate drastically if the calibration trials are insufficient. This paper develops a new scheme to learn from limited calibration data.a#13; alt;iagt;Approachalt;/iagt;: A learning across multiple stimuli scheme is proposed for the target recognition methods, which applies to learning the data corresponding to not only the target stimulus but also the other stimuli. The resulting optimization problems can be simplified and solved utilizing the prior knowledge and properties of SSVEPs across different stimuli. With the new learning scheme, the eCCA and the eTRCA can be extended to the multi-stimulus eCCA (ms-eCCA) and the multi-stimulus eTRCA (ms-eTRCA), respectively, as well as a combination of them (i.e., ms-eCCA + ms-eTRCA) that incorporates their merits.a#13; alt;iagt;Main resultsalt;/iagt;: Evaluation and comparison using an SSVEP-BCI benchmark dataset with 35 subjects show that the ms-eCCA (or ms-eTRCA) performs significantly better than the eCCA (or eTRCA) method while the ms-eCCA+ms-eTRCA performs the best. With the learning across stimuli scheme, the existing target recognition methods can be further improved in terms of the target recognition performance and the ability against insufficient calibration. a#13; alt;iagt;Significancealt;/iagt;: A new learning scheme is proposed towards efficient use of the calibration data, providing enhanced performance and saving calibration time in the SSVEP-based BCIs.

Comparing user-dependent and user-independent training of CNN for SSVEP BCI.

Abstract: OBJECTIVE We presented a comparative study on the training methodologies of a convolutional neural network (CNN) for the detection of steady-state visually evoked potentials (SSVEP). Two training scenarios were also compared: user-independent (UI) training and user-dependent (UD) training. APPROACH The CNN was trained in both UD and UI scenarios on two types of features for SSVEP classification: magnitude spectrum features (M-CNN) and complex spectrum features (C-CNN). The canonical correlation analysis (CCA), widely used in SSVEP processing, was used as the baseline. Additional comparisons were performed with task-related components analysis (TRCA) and filter-bank canonical correlation analysis (FBCCA). The performance of the proposed CNN pipelines, CCA, FBCCA and TRCA were evaluated with two datasets: a seven-class SSVEP dataset collected on 21 healthy participants and a twelve-class publicly available SSVEP dataset collected on ten healthy participants. MAIN RESULTS The UD based training methods consistently outperformed the UI methods when all other conditions were the same, as one would expect. However, the proposed UI-C-CNN approach performed similarly to the UD-M-CNN across all cases investigated on both datasets. On Dataset 1, the average accuracies of the different methods for 1 s window length were: CCA: 69.1% ± 10.8%, TRCA: 13.4% ± 1.5%, FBCCA: 64.8% ± 15.6%, UI-M-CNN: 73.5% ± 16.1%, UI-C-CNN: 81.6% ± 12.3%, UD-M-CNN: 87.8% ± 7.6% and UD-C-CNN: 92.5% ± 5%. On Dataset 2, the average accuracies of the different methods for data length of 1 s were: UD-C-CNN: 92.33% ± 11.1%, UD-M-CNN: 82.77% ± 16.7%, UI-C-CNN: 81.6% ± 18%, UI-M-CNN: 70.5% ± 22%, FBCCA: 67.1% ± 21%, CCA: 62.7% ± 21.5%, TRCA: 40.4% ± 14%. Using t-SNE, visualizing the features extracted by the CNN pipelines further revealed that the C-CNN method likely learned both the amplitude and phase related information from the SSVEP data for classification, resulting in superior performance than the M-CNN methods. The results suggested that UI-C-CNN method proposed in this study offers a good balance between performance and cost of training data. SIGNIFICANCE The proposed C-CNN based method is a suitable candidate for SSVEP-based BCIs and provides an improved performance in both UD and UI training scenarios.

Functional vagotopy in the cervical vagus nerve of the domestic pig: implications for the study of vagus nerve stimulation.

Abstract: Objective: Given current clinical interest in vagus nerve stimulation (VNS), there are surprisingly few studies characterizing the anatomy of the vagus nerve in large animal models as it pertains to on-and off-target engagement of local fibers. We sought to address this gap by evaluating vagal anatomy in the pig, whose vagus nerve organization and size approximates the human vagus nerve. Approach: Here we combined microdissection, histology, and immunohistochemistry to provide data on key features across the cervical vagus nerve in a swine model, and compare our results to other animal models (mouse, rat, dog, non-human primate) and humans. Main results: In a swine model we quantified the nerve diameter, number and diameter of fascicles, and distance of fascicles from the epineural surface where stimulating electrodes are placed. We also characterized the relative locations of the superior and recurrent laryngeal branches of the vagus nerve that have been implicated in therapy limiting side effects with common electrode placement. We identified key variants across the cohort that may be important for VNS with respect to changing sympathetic/parasympathetic tone, such as cross-connections to the sympathetic trunk. We discovered that cell bodies of pseudo-unipolar cells aggregate together to form a very distinct grouping within the nodose ganglion. This distinct grouping gives rise to a larger number of smaller fascicles as one moves caudally down the vagus nerve. This often leads to a distinct bimodal organization, or 'vagotopy'. This vagotopy was supported by immunohistochemistry where approximately half of the fascicles were immunoreactive for choline acetyltransferase, and reactive fascicles were generally grouped in one half of the nerve. Significance: The vagotopy observed via histology may be advantageous to exploit in design of electrodes/stimulation paradigms. We also placed our data in context of historic and recent histology spanning multiple models, thus providing a comprehensive resource to understand similarities and differences across species.

Supervised machine learning tools: a tutorial for clinicians

Abstract: In an increasingly data-driven world, artificial intelligence is expected to be a key tool for converting big data into tangible benefits and the healthcare domain is no exception to this. Machine learning aims to identify complex patterns in multi-dimensional data and use these uncovered patterns to classify new unseen cases or make data-driven predictions. In recent years, deep neural networks have shown to be capable of producing results that considerably exceed those of conventional machine learning methods for various classification and regression tasks. In this paper, we provide an accessible tutorial of the most important supervised machine learning concepts and methods, including deep learning, which are potentially the most relevant for the medical domain. We aim to take some of the mystery out of machine learning and depict how machine learning models can be useful for medical applications. Finally, this tutorial provides a few practical suggestions for how to properly design a machine learning model for a generic medical problem.

Open source silicon microprobes for high throughput neural recording.

Abstract: Objective Microfabricated multielectrode arrays are widely used for high throughput recording of extracellular neural activity, which is transforming our understanding of brain function in health and disease. Currently there is a plethora of electrode-based tools being developed at higher education and research institutions. However, taking such tools from the initial research and development phase to widespread adoption by the neuroscience community is often hindered by several obstacles. The objective of this work is to describe the development, application, and open dissemination of silicon microprobes for recording neural activity in vivo. Approach We propose an open source dissemination platform as an alternative to commercialization. This framework promotes recording tools that are openly and inexpensively available to the community. The silicon microprobes are designed in house, but the fabrication and assembly processes are carried out by third party companies. This enables mass production, a key requirement for large-scale dissemination. Main results We demonstrate the operation of silicon microprobes containing up to 256 electrodes in conjunction with optical fibers for optogenetic manipulations or fiber photometry. These data provide new insights about the relationship between calcium activity and neural spiking activity. We also describe the current state of dissemination of these tools. A file repository of resources related to designing, using, and sharing these tools is maintained online. Significance This paper is likely to be a valuable resource for both current and prospective users, as well as developers of silicon microprobes. Based on their extensive usage by a number of labs including ours, these tools present a promising alternative to other types of electrode-based technologies aimed at high throughput recording in head-fixed animals. This work also demonstrates the importance of validating fiber photometry measurements with simultaneous electrophysiological recordings.

Deep learning-based BCI for gait decoding from EEG with LSTM recurrent neural network.

Abstract: Objective Mobile Brain/Body Imaging (MoBI) frameworks allowed the research community to find evidence of cortical involvement at walking initiation and during locomotion. However, the decoding of gait patterns from brain signals remains an open challenge. The aim of this work is to propose and validate a deep learning model to decode gait phases from Electroenchephalography (EEG). Approach A Long-Short Term Memory (LSTM) deep neural network has been trained to deal with time-dependent information within brain signals during locomotion. The EEG signals have been preprocessed by means of Artifacts Subspace Reconstruction (ASR) and Reliable Independent Component Analysis (RELICA) to ensure that classification performance was not affected by movement-related artifacts. Main results The network was evaluated on the dataset of 11 healthy subjects walking on a treadmill. The proposed decoding approach shows a robust reconstruction (AUC > 90%) of gait patterns (i.e. swing and stance states) of both legs together, or of each leg independently. Significance Our results support for the first time the use of a memory-based deep learning classifier to decode walking activity from non-invasive brain recordings. We suggest that this classifier, exploited in real time, can be a more effective input for devices restoring locomotion in impaired people.

An online SSVEP-BCI system in an optical see-through augmented reality environment.

Abstract: Objective This study aimed to design and evaluate a high-speed online steady-state visually evoked potential (SSVEP)-based brain-computer interface (BCI) in an optical see-through (OST) augmented reality (AR) environment. Approach An eight-class BCI was designed in an OST-AR headset which is wearable and allows users to see the user interface of the BCI and the device to be controlled in the same view field via the OST head-mounted display. The accuracies, information transfer rates (ITRs), and SSVEP signal characteristics of the AR-BCI were evaluated and compared with a computer screen-based BCI implemented with a laptop in offline and online cue-guided tasks. Then, the performance of the AR-BCI was evaluated in an online robotic arm control task. Main results The offline results obtained during the cue-guided task performed with the AR-BCI showed maximum averaged ITRs of 65.50 ± 9.86 bits min-1 according to the extended canonical correlation analysis-based target identification method. The online cue-guided task achieved averaged ITRs of 65.03 ± 11.40 bits min-1. The online robotic arm control task achieved averaged ITRs of 45.57 ± 7.40 bits min-1. Compared with the screen-based BCI, some limitations of the AR environment impaired BCI performance and the quality of SSVEP signals. Significance The results showed the potential for providing a high-performance brain-control interaction method by combining AR and BCI. This study could provide methodological guidelines for developing more wearable BCIs in OST-AR environments and will also encourage more interesting applications involving BCIs and AR techniques.

Graph theory analysis of directed functional brain networks in major depressive disorder based on EEG signal.

Abstract: Objective. Analysis of functional and structural brain networks have suggested that major depressive disorder (MDD) is associated with a disruption in brain networks. This paper aims to investigate the abnormalities of brain networks in MDD. Approach. To this aim, we constructed weighted directed functional networks based on Electroencephalography (EEG) signals of 26 MDD patients and 23 normal (N) subjects. The nodes of networks were 19 EEG electrodes, and the edges were phase transfer entropy (PTE) between each pair of electrodes. PTE is a model-free, phase-based effective connectivity measure that is relatively robust to noise and linear mixing. Since the correct instantaneous phase of a signal is computed for narrow frequency bands, the networks were analyzed in eight frequency sub-bands including delta, theta, alpha1, alpha2, beta1, beta2, beta3, and beta4. To assess the alteration in the topology of brain networks in MDD patients, graph theory metrics consist of global efficiency (GE), local efficiency (LE), node betweenness centrality (BC), node degree, and node strength were analyzed by statistical test and classification. Furthermore, directed differential connectivity graph (dDCG) for MDD and N group was studied. Main results. These analyses revealed a higher node degree and strength in dDCG of MDD than normal. It was also found that MDD brain networks have a more randomized structure than the N group. Moreover, our result indicated the out-degree of networks classified MDD and N subjects with an accuracy of 92%; thus, it can be considered as a powerful feature for depression detection. Significance. Our analysis may provide new insights into developing biomarkers for depression detection based on brain networks. This study has the approval of the Iran University of Medical Sciences' ethics committee with 1397127 trial registration number.

Sources of off-target effects of vagus nerve stimulation using the helical clinical lead in domestic pigs.

Abstract: Objective Clinical data suggest that efficacious vagus nerve stimulation (VNS) is limited by side effects such as cough and dyspnea that have stimulation thresholds lower than those for therapeutic outcomes VNS side effects are putatively caused by activation of nearby muscles within the neck, via direct muscle activation or activation of nerve fibers innervating those muscles Our goal was to determine the thresholds at which various VNS-evoked effects occur in the domestic pig—an animal model with vagus anatomy similar to human—using the bipolar helical lead deployed clinically Approach Intrafascicular electrodes were placed within the vagus nerve to record electroneurographic (ENG) responses, and needle electrodes were placed in the vagal-innervated neck muscles to record electromyographic (EMG) responses Main results Contraction of the cricoarytenoid muscle occurred at low amplitudes (~03 mA) and resulted from activation of motor nerve fibers in the cervical vagus trunk within the electrode cuff which bifurcate into the recurrent laryngeal branch of the vagus At higher amplitudes (~14 mA), contraction of the cricoarytenoid and cricothyroid muscles was generated by current leakage outside the cuff to activate motor nerve fibers running within the nearby superior laryngeal branch of the vagus Activation of these muscles generated artifacts in the ENG recordings that may be mistaken for compound action potentials representing slowly conducting Aδ-, B-, and C-fibers Significance Our data resolve conflicting reports of the stimulation amplitudes required for C-fiber activation in large animal studies (>10 mA) and human studies (<250 μA) After removing muscle-generated artifacts, ENG signals with post-stimulus latencies consistent with Aδ- and B-fibers occurred in only a small subset of animals, and these signals had similar thresholds to those that caused bradycardia By identifying specific neuroanatomical pathways that cause off-target effects and characterizing the stimulation dose-response curves for on- and off-target effects, we hope to guide interpretation and optimization of clinical VNS

Continuous low-frequency EEG decoding of arm movement for closed-loop, natural control of a robotic arm.

Abstract: Objective Continuous decoding of voluntary movement is desirable for closed-loop, natural control of neuroprostheses. Recent studies showed the possibility to reconstruct the hand trajectories from low-frequency (LF) electroencephalographic (EEG) signals. So far this has only been performed offline. Here, we attempt for the first time continuous online control of a robotic arm with LF-EEG-based decoded movements. Approach The study involved ten healthy participants, asked to track a moving target by controlling a robotic arm. At the beginning of the experiment, the robot was fully controlled by the participant's hand trajectories. After calibrating the decoding model, that control was gradually replaced by LF-EEG-based decoded trajectories, first with 33%, 66% and finally 100% EEG control. Likewise with other offline studies, we regressed the movement parameters (two-dimensional positions, velocities, and accelerations) from the EEG with partial least squares (PLS) regression. To integrate the information from the different movement parameters, we introduced a combined PLS and Kalman filtering approach (named PLSKF). Main results We obtained moderate yet overall significant (α = 0.05) online correlations between hand kinematics and PLSKF-decoded trajectories of 0.32 on average. With respect to PLS regression alone, the PLSKF had a stable correlation increase of Δr = 0.049 on average, demonstrating the successful integration of different models. Parieto-occipital activations were highlighted for the velocity and acceleration decoder patterns. The level of robot control was above chance in all conditions. Participants finally reported to feel enough control to be able to improve with training, even in the 100% EEG condition. Significance Continuous LF-EEG-based movement decoding for the online control of a robotic arm was achieved for the first time. The potential bottlenecks arising when switching from offline to online decoding, and possible solutions, were described. The effect of the PLSKF and its extensibility to different experimental designs were discussed.

Towards emerging EEG applications: a novel printable flexible Ag/AgCl dry electrode array for robust recording of EEG signals at forehead sites.

Abstract: Objectives. With the rapid development of EEG-based wearable healthcare devices and brain-computer interfaces (BCIs), reliable and user-friendly EEG sensors for EEG recording especially at the forehead sites are highly desirable and challenging. However, the existing EEG sensors cannot meet the requirements, since wet electrodes require tedious setup and conductive pastes or gels, and most dry electrodes show unacceptable high contact impedance. In addition, the existing electrodes cannot absorb sweat effectively, which would cause cross-interferences even short circuits between adjacent electrodes, especially in the moving scenarios, or hot and humid environment. To resolve these problems, a novel printable flexible Ag/AgCl dry electrode array was developed for EEG acquisition at forehead sites, mainly consisting of screen-printing Ag/AgCl coating, conductive sweat-absorbable sponges and flexible tines. Approach. A systematic method was also established to evaluate the flexible dry electrode array. Main results. The experimental results show the flexible dry electrode array have reproducible electrode potential, relative low electrode-skin impedance, and good stability. Moreover, the EEG signals can be effectively captured with high quality that is comparable with that of wet electrodes. Significance. All the results confirmed that the feasibility for forehead EEG recording in real-world scenarios using the proposed flexible dry electrode array, with a rapid and facile operation as well as advantages of self-application, user-friendliness and wearing comfort.

Spike detection and sorting with deep learning.

Abstract: Objective The extraction and identification of single-unit activities in intracortically recorded electric signals have a key role in basic neuroscience, but also in applied fields, like in the development of high-accuracy brain-computer interfaces. The purpose of this paper is to present our current results on the detection, classification and prediction of neural activities based on multichannel action potential recordings. Approach Throughout our investigations, a deep learning approach utilizing convolutional neural networks and a combination of recurrent and convolutional neural networks was applied, with the latter used in case of spike detection and the former used for cases of sorting and predicting spiking activities. Main results In our experience, the algorithms applied prove to be useful in accomplishing the tasks mentioned above: our detector could reach an average recall of 69%, while we achieved an average accuracy of 89% in classifying activities produced by more than 20 distinct neurons. Significance Our findings support the concept of creating real-time, high-accuracy action potential based BCIs in the future, providing a flexible and robust algorithmic background for further development.

Ultra-small Carbon Fiber Electrode Recording Site Optimization and Improved in Vivo Chronic Recording Yield

Abstract: Objective Carbon fiber electrodes may enable better long-term brain implants, minimizing the tissue response commonly seen with silicon-based electrodes. The small diameter fiber may enable high-channel count brain-machine interfaces capable of reproducing dexterous movements. Past carbon fiber electrodes exhibited both high fidelity single unit recordings and a healthy neuronal population immediately adjacent to the recording site. However, the recording yield of our carbon fiber arrays chronically implanted in the brain typically hovered around 30%, for previously unknown reasons. In this paper we investigated fabrication process modifications aimed at increasing recording yield and longevity. Approach We tested a new cutting method using a 532nm laser against traditional scissor methods for the creation of the electrode recording site. We verified the efficacy of improved recording sites with impedance measurements and in vivo array recording yield. Additionally, we tested potentially longer-lasting coating alternatives to PEDOT:pTS, including PtIr and oxygen plasma etching. New coatings were evaluated with accelerated soak testing and acute recording. Main results We found that the laser created a consistent, sustainable 257 ± 13.8 µm2 electrode with low 1 kHz impedance (19 ± 4 kΩ with PEDOT:pTS) and low fiber-to-fiber variability. The PEDOT:pTS coated laser cut fibers were found to have high recording yield in acute (97% > 100 µV pp , N = 34 fibers) and chronic (84% > 100 µV pp , day 7; 71% > 100 µV pp , day 63, N = 45 fibers) settings. The laser cut recording sites were good platforms for the PtIr coating and oxygen plasma etching, slowing the increase in 1 kHz impedance compared to PEDOT:pTS in an accelerated soak test. Significance We have found that laser cut carbon fibers have a high recording yield that can be maintained for over two months in vivo and that alternative coatings perform better than PEDOT:pTS in accelerated aging tests. This work provides evidence to support carbon fiber arrays as a viable approach to high-density, clinically-feasible brain-machine interfaces.

Data augmentation for self-paced motor imagery classification with C-LSTM.

Abstract: Objective. Brain Computer Interfaces (BCI) are becoming important tools for assistive technology, particularly through the use of Motor Imagery (MI) for aiding task completion. However, most existing methods of MI classification have been applied in a trial-wise fashion, with window sizes of approximately 2 seconds or more. Application of this type of classifier could cause a delay when switching between MI events. Approach. In this study, state-of-the-art classification methods for motor imagery are assessed with considerations for real-time and self-paced control, and a Convolutional Long-Short Term Memory (C-LSTM) network based on Filter Bank Common Spatial Patterns (FBCSP) is proposed. In addition, the effects of several methods of data augmentation on different classifiers are explored. Main Results. The results of this study show that the proposed network achieves adequate results in distinguishing between different control classes, but both considered deep learning models are still less reliable than a Riemannian Minimum Distance to the Mean (MDM) classifier. In addition, controlled skewing of the data and the explored data augmentation methods improved the average overall accuracy of the classifiers by 14.0% and 5.3%, respectively. Significance. This manuscript is among the first to attempt combining convolutional and recurrent neural network layers for the purpose of MI classification, and is also one of the first to provide an in-depth comparison of various data augmentation methods for MI classification. In addition, all of these methods are applied on smaller windows of data and with consideration to ambient data, which provides a more realistic test bed for real-time and self-paced control.

Decoding spoken English from intracortical electrode arrays in dorsal precentral gyrus.

Abstract: Objective To evaluate the potential of intracortical electrode array signals for brain-computer interfaces (BCIs) to restore lost speech, we measured the performance of decoders trained to discriminate a comprehensive basis set of 39 English phonemes and to synthesize speech sounds via a neural pattern matching method. We decoded neural correlates of spoken-out-loud words in the 'hand knob' area of precentral gyrus, a step toward the eventual goal of decoding attempted speech from ventral speech areas in patients who are unable to speak. Approach Neural and audio data were recorded while two BrainGate2 pilot clinical trial participants, each with two chronically-implanted 96-electrode arrays, spoke 420 different words that broadly sampled English phonemes. Phoneme onsets were identified from audio recordings, and their identities were then classified from neural features consisting of each electrode's binned action potential counts or high-frequency local field potential power. Speech synthesis was performed using the 'Brain-to-Speech' pattern matching method. We also examined two potential confounds specific to decoding overt speech: acoustic contamination of neural signals and systematic differences in labeling different phonemes' onset times. Main results A linear decoder achieved up to 29.3% classification accuracy (chance = 6%) across 39 phonemes, while an RNN classifier achieved 33.9% accuracy. Parameter sweeps indicated that performance did not saturate when adding more electrodes or more training data, and that accuracy improved when utilizing time-varying structure in the data. Microphonic contamination and phoneme onset differences modestly increased decoding accuracy, but could be mitigated by acoustic artifact subtraction and using a neural speech onset marker, respectively. Speech synthesis achieved r = 0.523 correlation between true and reconstructed audio. Significance The ability to decode speech using intracortical electrode array signals from a nontraditional speech area suggests that placing electrode arrays in ventral speech areas is a promising direction for speech BCIs.

Impact of parameter selection on estimates of motoneuron excitability using paired motor unit analysis.

Abstract: Objective Noninvasive estimation of motoneuron excitability in human motoneurons is achieved through a paired motor unit analysis (ΔF) that quantifies hysteresis in the instantaneous firing rates at motor unit recruitment and de-recruitment. The ΔF technique provides insight into the magnitude of neuromodulatory synaptic input and persistent inward currents (PICs). While the ΔF technique is commonly used for estimating motoneuron excitability during voluntary contractions, computational parameters used for the technique vary across studies. A systematic investigation into the relationship between these parameters and ΔF values is necessary. Approach We assessed the sensitivity of the ΔF technique with several criteria commonly used in selecting motor unit pairs for analysis and methods used for smoothing the instantaneous motor unit firing rates. Using high-density surface EMG and convolutive blind source separation, we obtained a large number of motor unit pairs (5409) from the triceps brachii of ten healthy individuals during triangular isometric contractions. Main results We found an exponential plateau relationship between ΔF and the recruitment time difference between the motor unit pairs and an exponential decay relationship between ΔF and the de-recruitment time difference between the motor unit pairs, with the plateaus occurring at approximately 1 s and 1.5 s, respectively. Reduction or removal of the minimum threshold for rate-rate correlation of the two units did not affect ΔF values or variance. Removing motor unit pairs in which the firing rate of the control unit was saturated had no significant effect on ΔF. Smoothing the filter selection had no substantial effect on ΔF values and ΔF variance; however, filter selection affected the minimum recruitment and de-recruitment time differences. Significance Our results offer recommendations for standardized parameters for the ΔF approach and facilitate the interpretation of findings from studies that implement the ΔF analysis but use different computational parameters.

Multi-channel EEG epileptic spike detection by a new method of tensor decomposition.

Abstract: Objective Epilepsy is one of the most common brain disorders. For epilepsy diagnosis or treatment, the neurologist needs to observe epileptic spikes from electroencephalography (EEG) data. Since multi-channel EEG records can be naturally represented by multi-way tensors, it is of interest to see whether tensor decomposition is able to analyze EEG epileptic spikes. Approach In this paper, we first proposed the problem of simultaneous multilinear low-rank approximation of tensors (SMLRAT) and proved that SMLRAT can obtain local optimum solutions by using two well-known tensor decomposition algorithms (HOSVD and Tucker-ALS). Second, we presented a new system for automatic epileptic spike detection based on SMLRAT. Main results We propose to formulate the problem of feature extraction from a set of EEG segments, represented by tensors, as the SMLRAT problem. Efficient EEG features were obtained, based on estimating the 'eigenspikes' derived from nonnegative GSMLRAT. We compared the proposed tensor analysis method with other common tensor methods in analyzing EEG signal and compared the proposed feature extraction method with the state-of-the-art methods. Experimental results indicated that our proposed method is able to detect epileptic spikes with high accuracy. Significance Our method, for the first time, makes a step forward for automatic detection EEG epileptic spikes based on tensor decomposition. The method can provide a practical solution to distinguish epileptic spikes from artifacts in real-life EEG datasets.

Deep brain stimulation: a review of the open neural engineering challenges

Abstract: Objective: Deep brain stimulation (DBS) is an established and valid therapy for a variety of pathological conditions ranging from motor to cognitive disorders. Still, much of the DBS-related mechanism of action is far from being understood, and there are several side effects of DBS whose origin is unclear. In the last years DBS limitations have been tackled by a variety of approaches, including adaptive deep brain stimulation (aDBS), a technique that relies on using chronically implanted electrodes on "sensing mode" to detect the neural markers of specific motor symptoms and to deliver on-demand or modulate the stimulation parameters accordingly. Here we will review the state of the art of the several approaches to improve DBS and summarize the main challenges toward the development of an effective aDBS therapy. Approach: we discuss models of basal ganglia disorders pathogenesis, hardware and software improvements for conventional DBS, and candidate neural and non-neural features and related control strategies for aDBS. We identify then the main operative challenges toward optimal DBS such as (i) accurate target localization, (ii) increased spatial resolution of stimulation, (iii) development of in silico tests for DBS, (iv) identification of specific motor symptoms biomarkers, in particular (v) assessing how LFP oscillations relate to behavioral disfunctions, and (vi) clarify how stimulation affects the cortico-basal-ganglia-thalamic network to (vii) design optimal stimulation patterns. Significance: This roadmap will lead neural engineers novel to the field toward the most relevant open issues of DBS, while the in-depth readers might find a careful comparison of advantages and drawbacks of the most recent attempts to improve DBS-related neuromodulatory strategies.

Thinker invariance: enabling deep neural networks for BCI across more people.

Abstract: OBJECTIVE Most deep neural networks (DNNs) used as brain computer interfaces (BCI) classifiers are rarely viable for more than one person and are relatively shallow compared to the state-of-the-art in the wider machine learning literature. The goal of this work is to frame these as a unified challenge and reconsider how transfer learning is used to overcome these difficulties. APPROACH We present two variations of a holistic approach to transfer learning with DNNs for BCI that rely on a deeper network called TIDNet. Our approaches use multiple subjects for training in the interest of creating a more universal classifier that is applicable for new (unseen) subjects. The first approach is purely subject-invariant and the second targets specific subjects, without loss of generality. We use five publicly accessible datasets covering a range of tasks and compare our approaches to state-of-the-art alternatives in detail. MAIN RESULTS We observe that TIDNet in conjunction with our training augmentations is more consistent when compared to shallower baselines, and in some cases exhibits large and significant improvements, for instance motor imagery classification improvements of over 8%. Furthermore, we show that our suggested multi-domain learning (MDL) strategy strongly outperforms simply fine-tuned general models when targeting specific subjects, while remaining more generalizable to still unseen subjects. SIGNIFICANCE TIDNet in combination with a data alignment-based training augmentation proves to be a consistent classification approach of single raw trials and can be trained even with the inclusion of corrupted trials. Our MDL strategy calls into question the intuition to fine-tune trained classifiers to new subjects, as it proves simpler and more accurate while remaining general. Furthermore, we show evidence that augmented TIDNet training makes better use of additional subjects, showing continued and greater performance improvement over shallower alternatives, indicating promise for a new subject-invariant paradigm rather than a subject-specific one.

Motor imagery recognition with automatic EEG channel selection and deep learning.

Abstract: Objective Modern motor imagery (MI) -based brain computer interface (BCI) systems often entail a large number of electroencephalogram (EEG) recording channels. However, irrelevant or highly correlated channels would diminish the discriminatory ability, thus reducing the control capability of external devices. How to optimally select channels and extract associated features remains a big challenge. This study aims to propose and validate a deep learning-based approach to automatically recognize two different MI states by selecting the relevant EEG channels. Approach In this work, we make use of a sparse squeeze-and-excitation module to extract the weights of EEG channels based on their contribution to MI classification, by which an automatic channel selection (ACS) strategy is developed. Further, we propose a convolutional neural network (CNN) to fully exploit the time-frequency features, thus outperforming traditional classification methods both in terms of accuracy and robustness. Main results We execute the experiments using EEG signals recorded at MI where twenty-five healthy subjects performed MI movements of the right hand and feet to generate motor commands. An average accuracy of 87.2 ± 5.0 % (mean ± std) is obtained, providing a 37.3 % improvement with respect to a state-of-the-art channel selection approach. Significance The proposed automatic channel selection method has been found to be significantly advantageous compared to the typical approach of using a fixed channel configuration. This work shows that fewer EEG channels not only reduces computational complexity but also improves the MI classification performance. The proposed method selects EEG channels related to hand and feet movement, which paves the way to real-time and more natural interfaces between the patient and the robotic device.

Interfacing the neural output of the spinal cord: robust and reliable longitudinal identification of motor neurons in humans.

Abstract: Objective: Non-invasive electromyographic techniques can detect action potentials from muscle units with high spatial dimensionality. These technologies allow the decoding of large samples of motor units by using high-density grids of electrodes that are placed on the skin overlying contracting muscles and therefore provide a non-invasive representation of the human spinal cord output. Approach: From a sample of >1200 decoded motor neurons, we show that motor neuron activity can be identified in humans in the full muscle recruitment range with high accuracy. Main results: After showing the validity of decomposition with novel test parameters, we demonstrate that the same motor neurons can be tracked over a period of one-month, which allows for the longitudinal analysis of individual human neural cells. Significance: These results show the potential of an accurate and reliable assessment of large populations of motor neurons in physiological investigations. We discuss the potential of this non-invasive neural interfacing technology for the study of the neural determinants of movement and man-machine interfacing.

Stability of flexible thin-film metallization stimulation electrodes: analysis of explants after first-in-human study and improvement of in vivo performance.

Abstract: Objective. Micro-fabricated neural interfaces based on polyimide (PI) are achieving increasing importance in translational research. The ability to produce well-defined micro-structures with properties that include chemical inertness, mechanical flexibility and low water uptake are key advantages for these devices. Approach. This paper reports the development of the transverse intrafascicular multichannel electrode (TIME) used to deliver intraneural sensory feedback to an upper-limb amputee in combination with a sensorized hand prosthesis. A failure mode analysis on the explanted devices was performed after a first-inhuman study limited to 30 d. Main results. About 90% of the stimulation contact sites of the TIMEs maintained electrical functionality and stability during the full implant period. However, optical analysis post-explantation revealed that 62.5% of the stimulation contacts showed signs of delamination at the metallization-PI interface. Such damage likely occurred due to handling during explantation and subsequent analysis, since a significant change in impedance was not observed in vivo. Nevertheless, whereas device integrity is mandatory for long-term functionality in chronic implantation, measures to increase the bonding strength of the metallization-PI interface deserve further investigation. We report here that silicon carbide (SiC) is an effective adhesion-promoting layer resisting heavy electrical stimulation conditions within a rodent animal trial. Optical analysis of the new electrodes revealed that the metallization remained unaltered after delivering over 14 million pulses in vivo without signs of delamination at the metallization-PI interface. Significance. Failure mode analysis guided implant © 2020 IOP Publishing Ltd J. Neural Eng. 17 (2020) 046006 PCvancara et al stability optimization. Reliable adhesion of thin-film metallization to substrate has been proven using SiC, improving the potential transfer of micro-fabricated neural electrodes for chronic clinical applications. (Document number of Ethical Committee: P/905/CE/2012; Date of approval: 2012-10-04)