Showing papers in "Journal of Neural Engineering in 2021"
TL;DR: The frontiers of applying deep learning for non-invasive brain signals analysis are provided, by summarizing a large number of recent publications, and the potential real-world applications which benefit not only disabled people but also normal individuals are reported.
Abstract: Brain signals refer to the biometric information collected from the human brain. The research on brain signals aims to discover the underlying neurological or physical status of the individuals by signal decoding. The emerging deep learning techniques have improved the study of brain signals significantly in recent years. In this work, we first present a taxonomy of non-invasive brain signals and the basics of deep learning algorithms. Then, we provide a comprehensive survey of the frontiers of applying deep learning for non-invasive brain signals analysis, by summarizing a large number of recent publications. Moreover, upon the deep learning-powered brain signal studies, we report the potential real-world applications which benefit not only disabled people but also normal individuals. Finally, we discuss the opening challenges and future directions.
128 citations
TL;DR: The results suggest that self-supervision may pave the way to a wider use of deep learning models on EEG data, and linear classifiers trained on SSL-learned features consistently outperformed purely supervised deep neural networks in low-labeled data regimes while reaching competitive performance when all labels were available.
Abstract: Objective. Supervised learning paradigms are often limited by the amount of labeled data that is available. This phenomenon is particularly problematic in clinically-relevant data, such as electroencephalography (EEG), where labeling can be costly in terms of specialized expertise and human processing time. Consequently, deep learning architectures designed to learn on EEG data have yielded relatively shallow models and performances at best similar to those of traditional feature-based approaches. However, in most situations, unlabeled data is available in abundance. By extracting information from this unlabeled data, it might be possible to reach competitive performance with deep neural networks despite limited access to labels. Approach. We investigated self-supervised learning (SSL), a promising technique for discovering structure in unlabeled data, to learn representations of EEG signals. Specifically, we explored two tasks based on temporal context prediction as well as contrastive predictive coding on two clinically-relevant problems: EEG-based sleep staging and pathology detection. We conducted experiments on two large public datasets with thousands of recordings and performed baseline comparisons with purely supervised and hand-engineered approaches. Main results. Linear classifiers trained on SSL-learned features consistently outperformed purely supervised deep neural networks in low-labeled data regimes while reaching competitive performance when all labels were available. Additionally, the embeddings learned with each method revealed clear latent structures related to physiological and clinical phenomena, such as age effects. Significance. We demonstrate the benefit of SSL approaches on EEG data. Our results suggest that self-supervision may pave the way to a wider use of deep learning models on EEG data.
113 citations
TL;DR: An overview of existing methods for MT-BCI user training is provided, a categorization and taxonomy of these training approaches are presented, guidelines on how to choose the best methods are provided and open challenges and perspectives are identified to further improve user training.
Abstract: Mental-Tasks based Brain-Computer Interfaces (MT-BCIs) allow their users to interact with an external device solely by using brain signals produced through mental tasks. While MT-BCIs are promising for many applications, they are still barely used outside laboratories due to their lack of reliability. MT-BCIs require their users to develop the ability to self-regulate specific brain signals. However, the human learning process to control a BCI is still relatively poorly understood and how to optimally train this ability is currently under investigation. Despite their promises and achievements, traditional training programs have been shown to be sub-optimal and could be further improved. In order to optimize user training and improve BCI performance, human factors should be taken into account. An interdisciplinary approach should be adopted to provide learners with appropriate and/or adaptive training. In this article, we provide an overview of existing methods for MT-BCI user training - notably in terms of environment, instructions, feedback and exercises. We present a categorization and taxonomy of these training approaches, provide guidelines on how to choose the best methods and identify open challenges and perspectives to further improve MT-BCI user training.
62 citations
TL;DR: This study proposed a new training-free dynamical optimization algorithm, which significantly improved the performance of online SSVEP-based BCI systems and significantly outperforms STE-DW and FBCCA-FW in terms of accuracy and ITR.
Abstract: Objective: Filter bank canonical correlation analysis (FBCCA) is a widely-used classification approach implemented in steady-state visual evoked potential (SSVEP)–based brain-computer interfaces (BCIs). However, conventional detection algorithms for SSVEP recognition problems, including the FBCCA, were usually based on 'fixed window' strategy. That's to say, these algorithms always analyze data with fixed length. This study devoted to enhance the performance of SSVEP-based BCIs by designing a new dynamic window strategy which automatically finds an optimal data length to achieve higher information transfer rate (ITR). Approach: The main purpose of 'dynamic window' is to minimize the required data length while maintaining high accuracy. This study projected the correlation coefficients of FBCCA into probability space by softmax function and built a hypothesis testing model, which took risk function as evaluation of classification result's 'credibility'. In order to evaluate the superiority of this approach, FBCCA with fixed data length (FBCCA-FW) and spatial temporal equalization dynamic window (STE-DW) were implemented for comparison. Main results: Fourteen healthy subjects' results were concluded by a 40-target online SSVEP-based BCI speller system. The results suggest that this proposed approach significantly outperforms STE-DW and FBCCA-FW in terms of accuracy and ITR. Significance: By incorporating the fundamental ideas of FBCCA and dynamic window strategy, this study proposed a new training-free dynamical optimization algorithm, which significantly improved the performance of online SSVEP-based BCI systems.
53 citations
TL;DR: The EEGdenoiseNet as discussed by the authors is a benchmark EEG dataset that is suited for training and testing DL-based denoising models, as well as for performance comparisons across models.
Abstract: Objective.Deep learning (DL) networks are increasingly attracting attention across various fields, including electroencephalography (EEG) signal processing. These models provide comparable performance to that of traditional techniques. At present, however, there is a lack of well-structured and standardized datasets with specific benchmark limit the development of DL solutions for EEG denoising.Approach.Here, we present EEGdenoiseNet, a benchmark EEG dataset that is suited for training and testing DL-based denoising models, as well as for performance comparisons across models. EEGdenoiseNet contains 4514 clean EEG segments, 3400 ocular artifact segments and 5598 muscular artifact segments, allowing users to synthesize contaminated EEG segments with the ground-truth clean EEG.Main results.We used EEGdenoiseNet to evaluate denoising performance of four classical networks (a fully-connected network, a simple and a complex convolution network, and a recurrent neural network). Our results suggested that DL methods have great potential for EEG denoising even under high noise contamination.Significance.Through EEGdenoiseNet, we hope to accelerate the development of the emerging field of DL-based EEG denoising. The dataset and code are available athttps://github.com/ncclabsustech/EEGdenoiseNet.
47 citations
TL;DR: In this paper, a convolutional neural network (CNN) architecture for accurate and robust EEG-based motor imagery (MI) classification was proposed, which outperformed the state-of-the-art methods.
Abstract: Classification of EEG-based motor imagery (MI) is a crucial non-invasive application in brain-computer interface (BCI) research. This paper proposes a novel convolutional neural network (CNN) architecture for accurate and robust EEG-based MI classification that outperforms the state-of-the-art methods. The proposed CNN model, namely EEG-Inception, is built on the backbone of the Inception-Time network, which has showed to be highly efficient and accurate for time-series classification. Also, the proposed network is an end-to-end classification, as it takes the raw EEG signals as the input and does not require complex EEG signal-preprocessing. Furthermore, this paper proposes a novel data augmentation method for EEG signals to enhance the accuracy, at least by 3%, and reduce overfitting with limited BCI datasets. The proposed model outperforms all state-of-the-art methods by achieving the average accuracy of 88.4% and 88.6% on the 2008 BCI Competition IV 2a (four-classes) and 2b datasets (binary-classes), respectively. Furthermore, it takes less than 0.025 seconds to test a sample suitable for real-time processing. Moreover, the classification standard deviation for nine different subjects achieves the lowest value of 5.5 for the 2b dataset and 7.1 for the 2a dataset, which validates that the proposed method is highly robust. From the experiment results, it can be inferred that the EEG-Inception network exhibits a strong potential as a subject-independent classifier for EEG-based MI tasks.
44 citations
TL;DR: In this article, a polyacrylamide/polyvinyl alcohol superporous hydrogel (PAAm/PVA SPH)-based semi-dry electrode was constructed for capturing EEG signals at the hairy scalp, showing automatically "chargedischarge" electrolyte concept in EEG electrode development.
Abstract: A novel polyacrylamide/polyvinyl alcohol superporous hydrogel (PAAm/PVA SPH)-based semi-dry electrode was constructed for capturing EEG signals at the hairy scalp, showing automatically "charge-discharge" electrolyte concept in EEG electrode development. In this regard, PAAm/PVA SPH was polymerized in-situ in the hollow electrode cavity by freezing polymerization, which acted as a dynamic reservoir of electrolyte fluid. The superporous hydrogel can be completely "charged" with electrolyte fluid, such as saline, in just a few seconds and can be "discharged" through a few tiny pillars into the scalp at a desirable rate. In this way, an ideal local skin hydration effect was achieved at electrode-skin contact sites, facilitating the bioelectrical signal pathway and significantly reducing electrode-skin impedance. Moreover, the electrode interface effectively avoids short circuit and inconvenient issues. The results show that the semi-dry electrode displayed low and stable contact impedance, showing non-polarization properties with low off-set potential and negligible potential drift. The average temporal cross-correlation coefficient between the semi-dry and conventional wet electrodes was 0.941. Frequency spectra also showed almost identical responses with anticipated neural electrophysiology responses. Considering prominent advantages such as a rapid setup, robust signal, and user-friendliness, the new concept of semi-dry electrodes shows excellent potential in emerging real-life EEG applications.
43 citations
TL;DR: In this article, a literature review on surgical techniques for enhanced EMG interfaces, EMG sensors, decoding algorithms, and control paradigms for robotic lower limb prostheses is presented.
Abstract: Objective.Advanced robotic lower limb prostheses are mainly controlled autonomously. Although the existing control can assist cyclic movements during locomotion of amputee users, the function of these modern devices is still limited due to the lack of neuromuscular control (i.e. control based on human efferent neural signals from the central nervous system to peripheral muscles for movement production). Neuromuscular control signals can be recorded from muscles, called electromyographic (EMG) or myoelectric signals. In fact, using EMG signals for robotic lower limb prostheses control has been an emerging research topic in the field for the past decade to address novel prosthesis functionality and adaptability to different environments and task contexts. The objective of this paper is to review robotic lower limb Prosthesis control via EMG signals recorded from residual muscles in individuals with lower limb amputations.Approach.We performed a literature review on surgical techniques for enhanced EMG interfaces, EMG sensors, decoding algorithms, and control paradigms for robotic lower limb prostheses.Main results.This review highlights the promise of EMG control for enabling new functionalities in robotic lower limb prostheses, as well as the existing challenges, knowledge gaps, and opportunities on this research topic from human motor control and clinical practice perspectives.Significance.This review may guide the future collaborations among researchers in neuromechanics, neural engineering, assistive technologies, and amputee clinics in order to build and translate true bionic lower limbs to individuals with lower limb amputations for improved motor function.
42 citations
TL;DR: In this article, a calibration-free SSVEP-BCI system implemented 160 targets by four continuous sinusoidal stimuli that lasted four seconds in total, and the system achieved an average accuracy of 87.16 ± 11.46% and information transfer rate of 78.84 ± 15.59 bits min-1.
Abstract: Objective.Steady-state visual evoked potential (SSVEP) is an essential paradigm of electroencephalogram based brain-computer interface (BCI). Previous studies in the BCI research field mostly focused on enhancing classification accuracy and reducing stimuli duration. This study, however, concentrated on increasing the number of available targets in the BCI systems without calibration.Approach. Motivated by the idea of multiple frequency sequential coding, we developed a calibration-free SSVEP-BCI system implementing 160 targets by four continuous sinusoidal stimuli that lasted four seconds in total. Taking advantage of the benchmark dataset of SSVEP-BCI, this study optimized an arrangement of stimuli sequences, maximizing the response distance between different stimuli. We proposed an effective classification algorithm based on filter bank canonical correlation analysis. To evaluate the performance of this system, we conducted offline and online experiments using cue-guided selection tasks. Eight subjects participated in the offline experiments, and 12 subjects participated in the online experiments with real-time feedbacks.Mainresults. Offline experiments indicated the feasibility of the stimulation selection and detection algorithms. Furthermore, the online system achieved an average accuracy of 87.16 ± 11.46% and an information transfer rate of 78.84 ± 15.59 bits min-1. Specifically, seven of 12 subjects accomplished online experiments with accuracy higher than 90%. This study proposed an intact solution of applying numerous targets to SSVEP-based BCIs. Results of experiments confirmed the utility and efficiency of the system.Significance. This study firstly provides a calibration-free SSVEP-BCI speller system that enables more than 100 commands. This system could significantly expand the application scenario of SSVEP-based BCI. Meanwhile, the design criterion can hopefully enhance the overall performance of the BCI system. The demo video can be found in the supplementary material available online atstacks.iop.org/JNE/18/046094/mmedia.
39 citations
TL;DR: In this article, the authors evaluated the recording stability of implanted microelectrodes with iridium oxide film (SIROF) tips in the motor and somatosensory cortex of one person with spinal cord injury.
Abstract: Objective.Intracortical microstimulation (ICMS) in somatosensory cortex can restore sensation to people with spinal cord injury. However, the recording quality from implanted microelectrodes can degrade over time and limitations in stimulation longevity have been considered a potential barrier to the clinical use of ICMS. Our objective was to evaluate recording stability of intracortical electrodes implanted in the motor and somatosensory cortex of one person. The electrodes in motor cortex had platinum tips and were not stimulated, while the electrodes in somatosensory cortex had sputtered iridium oxide film (SIROF) tips and were stimulated. Additionally, we measured how well ICMS was able to evoke sensations over time.Approach. We implanted microelectrode arrays with SIROF tips in the somatosensory cortex (SIROF-sensory) of a human participant with a cervical spinal cord injury. We regularly stimulated these electrodes to evoke tactile sensations on the hand. Here, we quantify the stability of these electrodes in comparison to non-stimulated platinum electrodes implanted in the motor cortex (platinum-motor) over 1500 days with recorded signal quality and electrode impedances. Additionally, we quantify the stability of ICMS-evoked sensations using detection thresholds.Main results. We found that recording quality, as assessed by the number of electrodes with high-amplitude waveforms (>100µV peak-to-peak), peak-to-peak voltage, noise, and signal-to-noise ratio, decreased over time on SIROF-sensory and platinum-motor electrodes. However, SIROF-sensory electrodes were more likely to continue to record high-amplitude signals than platinum-motor electrodes. Interestingly, the detection thresholds for stimulus-evoked sensations decreased over time from a median of 31.5μA at day 100-10.4μA at day 1500, with the largest changes occurring between day 100 and 500.Significance. These results demonstrate that ICMS in human somatosensory cortex can be provided over long periods of time without deleterious effects on recording or stimulation capabilities. In fact, the sensitivity to stimulation improved over time.
32 citations
TL;DR: In this article, the authors report the potentialities and pitfalls of one of the first commercially available devices capable of recording brain local field potentials (LFPs) from the implanted DBS leads, chronically and during stimulation.
Abstract: Objective. Technical advances in deep brain stimulation (DBS) are crucial to improve therapeutic efficacy and battery life. We report the potentialities and pitfalls of one of the first commercially available devices capable of recording brain local field potentials (LFPs) from the implanted DBS leads, chronically and during stimulation. The aim was to provide clinicians with well-grounded tips on how to maximize the capabilities of this novel device, both in everyday practice and for research purposes.Approach. We collected clinical and neurophysiological data of the first 20 patients (14 with Parkinson's disease (PD), five with dystonia, one with chronic pain) that received the Percept™ PC in our centres. We also performed tests in a saline bath to validate the recordings quality.Main results. The Percept PC reliably recorded the LFP of the implanted site, wirelessly and in real time. We recorded the most promising clinically useful biomarkers for PD and dystonia (beta and theta oscillations) with and without stimulation. Furthermore, we provide an open-source code to facilitate export and analysis of data. Critical aspects of the system are presently related to contact selection, artefact detection, data loss, and synchronization with other devices.Significance. New technologies will soon allow closed-loop neuromodulation therapies, capable of adapting stimulation based on real-time symptom-specific and task-dependent input signals. However, technical aspects need to be considered to ensure reliable recordings. The critical use by a growing number of DBS experts will alert new users about the currently observed shortcomings and inform on how to overcome them.
TL;DR: A generalized transfer-learning framework for boosting the performance of steady-state visual evoked potential (SSVEP)-based brain–computer interfaces (BCIs) by leveraging cross-domain data transferring is established and significantly improved the SSVEP decoding accuracy over the standard TRCA approach when calibration data are limited.
Abstract: Objective: This study aims to establish a generalized transfer-learning framework for boosting the performance of steady-state visual evoked potential(SSVEP)-based brain-computer interfaces (BCIs) by leveraging cross-domain data transferring. Approach: We enhanced the state-of-the-art template-based SSVEP decoding through incorporating a least-squares transformation (LST)-based transfer learning to leverage calibration data across multiple domains (sessions, subjects, and EEG montages). Main results: Study results verified the efficacy of LST in obviating the variability of SSVEPs when transferring existing data across domains. Furthermore, the LST-based method achieved significantly higher SSVEP-decoding accuracy than the standard task-related component analysis (TRCA)-based method and the non-LST naive transfer-learning method. Significance: This study demonstrated the capability of the LST-based transfer learning to leverage existing data across subjects and/or devices with an in-depth investigation of its rationale and behavior in various circumstances. The proposed framework significantly improved the SSVEP decoding accuracy over the standard TRCA approach when calibration data are limited. Its performance in calibration reduction could facilitate plug-and-play SSVEP-based BCIs and further practical applications
TL;DR: The results demonstrate, for the first time, that using the proposed feature extraction method, it is possible to classify motor tasks from EEG recordings using a short interval of the data in the order of hundreds of milliseconds (e.g. 500 ms).
Abstract: Objective Classification of electroencephalography (EEG) signals with high accuracy using short recording intervals has been a challenging problem in developing brain computer interfaces (BCIs). This paper presents a novel feature extraction method for EEG recordings to tackle this problem. Approach The proposed approach is based on the concept that the brain functions in a dynamic manner, and utilizes dynamic functional connectivity graphs. The EEG data is first segmented into intervals during which functional networks sustain their connectivity. Functional connectivity networks for each identified segment are then localized, and graphs are constructed, which will be used as features. To take advantage of the dynamic nature of the generated graphs, a Long Short Term Memory (LSTM) classifier is employed for classification. Main results Features extracted from various durations of post-stimulus EEG data associated with motor execution and imagery tasks are used to test the performance of the classifier. Results show an average accuracy of 85.32% using features extracted from only 500 ms of the post-stimulus data. Significance Our results demonstrate, for the first time, that using the proposed feature extraction method, it is possible to classify motor tasks from EEG recordings using a short interval of the data in the order of hundreds of milliseconds (e.g. 500 ms). This duration is considerably shorter than what has been reported before. These results will have significant implications for improving the effectiveness and the speed of BCIs, particularly for those used in assistive technologies.
TL;DR: This work represents the first experimental application of state space model-based feedback control for optogenetic stimulation and should generalize to future problems involving the control of highly complex neural circuits.
Abstract: Objective.The rapid acceleration of tools for recording neuronal populations and targeted optogenetic manipulation has enabled real-time, feedback control of neuronal circuits in the brain. Continuously-graded control of measured neuronal activity poses a wide range of technical challenges, which we address through a combination of optogenetic stimulation and a state-space optimal control framework implemented in the thalamocortical circuit of the awake mouse.Approach.Closed-loop optogenetic control of neurons was performed in real-time via stimulation of channelrhodopsin-2 expressed in the somatosensory thalamus of the head-fixed mouse. A state-space linear dynamical system model structure was used to approximate the light-to-spiking input-output relationship in both single-neuron as well as multi-neuron scenarios when recording from multielectrode arrays. These models were utilized to design state feedback controller gains by way of linear quadratic optimal control and were also used online for estimation of state feedback, where a parameter-adaptive Kalman filter provided robustness to model-mismatch.Main results.This model-based control scheme proved effective for feedback control of single-neuron firing rate in the thalamus of awake animals. Notably, the graded optical actuation utilized here did not synchronize simultaneously recorded neurons, but heterogeneity across the neuronal population resulted in a varied response to stimulation. Simulated multi-output feedback control provided better control of a heterogeneous population and demonstrated how the approach generalizes beyond single-neuron applications.Significance.To our knowledge, this work represents the first experimental application of state space model-based feedback control for optogenetic stimulation. In combination with linear quadratic optimal control, the approaches laid out and tested here should generalize to future problems involving the control of highly complex neural circuits. More generally, feedback control of neuronal circuits opens the door to adaptively interacting with the dynamics underlying sensory, motor, and cognitive signaling, enabling a deeper understanding of circuit function and ultimately the control of function in the face of injury or disease.
TL;DR: In this paper, the authors developed a stretchable cuff electrode for low-voltage nerve stimulation, which combines favorable mechanical properties and good electrode performance with inert and stable materials, making it ideal for low power supply applications within bioelectronic medicine.
Abstract: Objective. Electrical stimulation of the peripheral nervous system (PNS) can treat various diseases and disorders, including the healing process after nerve injury. A major challenge when designing electrodes for PNS stimulation is the mechanical mismatch between the nerve and the device, which can lead to non-conformal contact, tissue damage and inefficient stimulation due to current leakage. Soft and stretchable cuff electrodes promise to tackle these challenges but often have limited performance and rely on unconventional materials. The aim of this study is to develop a high performance soft and stretchable cuff electrode based on inert materials for low-voltage nerve stimulation.Approach. We developed 50µm thick stretchable cuff electrodes based on silicone rubber, gold nanowire conductors and platinum coated nanowire electrodes. The electrode performance was characterized under strain cycling to assess the durability of the electrodes. The stimulation capability of the cuff electrodes was evaluated in anin vivosciatic nerve rat model by measuring the electromyography response to various stimulation pulses.Main results. The stretchable cuff electrodes showed excellent stability for 50% strain cycling and one million stimulation pulses. Saturated homogeneous stimulation of the sciatic nerve was achieved at only 200 mV due to the excellent conformability of the electrodes, the low conductor resistance (0.3 Ohm sq-1), and the low electrode impedance.Significance. The developed stretchable cuff electrode combines favourable mechanical properties and good electrode performance with inert and stable materials, making it ideal for low power supply applications within bioelectronic medicine.
TL;DR: In this article, the authors present the main clinical results achieved so far by different bioelectronic medicine approaches and discuss the challenges encountered in fully exploiting the potential of neuromodulatory strategies.
Abstract: Bioelectronic medicine (BM) is an emerging new approach for developing novel neuromodulation therapies for pathologies that have been previously treated with pharmacological approaches. In this review, we will focus on the neuromodulation of autonomic nervous system (ANS) activity with implantable devices, a field of BM that has already demonstrated the ability to treat a variety of conditions, from inflammation to metabolic and cognitive disorders. Recent discoveries about immune responses to ANS stimulation are the laying foundation for a new field holding great potential for medical advancement and therapies and involving an increasing number of research groups around the world, with funding from international public agencies and private investors. Here, we summarize the current achievements and future perspectives for clinical applications of neural decoding and stimulation of the ANS. First, we present the main clinical results achieved so far by different BM approaches and discuss the challenges encountered in fully exploiting the potential of neuromodulatory strategies. Then, we present current preclinical studies aimed at overcoming the present limitations by looking for optimal anatomical targets, developing novel neural interface technology, and conceiving more efficient signal processing strategies. Finally, we explore the prospects for translating these advancements into clinical practice.
TL;DR: The state-of-the-art supporting the development of a network theoretic approach as a promising tool for understanding BCIs and improve usability is provided.
Abstract: Brain-computer interfaces (BCIs) make possible to interact with the external environment by decoding the mental intention of individuals. BCIs can therefore be used to address basic neuroscience questions but also to unlock a variety of applications from exoskeleton control to neurofeedback (NFB) rehabilitation. In general, BCI usability critically depends on the ability to comprehensively characterize brain functioning and correctly identify the user's mental state. To this end, much of the efforts have focused on improving the classification algorithms taking into account localized brain activities as input features. Despite considerable improvement BCI performance is still unstable and, as a matter of fact, current features represent oversimplified descriptors of brain functioning. In the last decade, growing evidence has shown that the brain works as a networked system composed of multiple specialized and spatially distributed areas that dynamically integrate information. While more complex, looking at how remote brain regions functionally interact represents a grounded alternative to better describe brain functioning. Thanks to recent advances in network science, i.e. a modern field that draws on graph theory, statistical mechanics, data mining and inferential modelling, scientists have now powerful means to characterize complex brain networks derived from neuroimaging data. Notably, summary features can be extracted from these networks to quantitatively measure specific organizational properties across a variety of topological scales. In this topical review, we aim to provide the state-of-the-art supporting the development of a network theoretic approach as a promising tool for understanding BCIs and improve usability.
TL;DR: The lifetime of neural implant is strongly dependent on packaging due to the aqueous and biochemically aggressive nature of the body as mentioned in this paper, and the traditional approach is not amenable to the simultaneous goals of miniaturization, increased channel count, and wireless communication.
Abstract: The lifetime of neural implants is strongly dependent on packaging due to the aqueous and biochemically aggressive nature of the body. Over the last decade, there has been a drive towards neuromodulatory implants which are wireless and approaching millimeter-scales with increasing electrode count. A so-far unrealized goal for these new types of devices is an in-vivo lifetime comparable to a sizable fraction of a healthy patient's lifetime (>10-20 years). Existing, approved medical implants commonly package components in metal enclosures (e.g. titanium) with brazed ceramic inserts for electrode feedthrough. It is unclear how amenable the traditional approach is to the simultaneous goals of miniaturization, increased channel count, and wireless communication. Ceramic materials have also played a significant role in traditional medical implants due to their dielectric properties, corrosion resistance, biocompatibility, and high strength, but are not as commonly used for housing materials due to their brittleness and the difficulty they present in creating complex housing geometries. However, thin film technology has opened new opportunities for ceramics processing. Thin films derived largely from the semiconductor industry can be deposited and patterned in new ways, have conductivities which can be altered during manufacturing to provide conductors as well as insulators, and can be used to fabricate flexible substrates. In this review, we give an overview of packaging for neural implants, with an emphasis on how ceramic materials have been utilized in medical device packaging, as well as how ceramic thin film micromachining and processing may be further developed to create truly reliable, miniaturized, neural implants.
TL;DR: In this paper, transfer learning was used to adapt a classifier that was initially trained on intracranial electroencephalography (iEEG) signals to facilitate classification of non-EEG physiological datasets comprising accelerometry, blood volume pulse, skin electrodermal activity, heart rate, and temperature signals.
Abstract: Objective. The detection of seizures using wearable devices would improve epilepsy management, but reliable detection of seizures in an ambulatory environment remains challenging, and current studies lack concurrent validation of seizures using electroencephalography (EEG) data.Approach. An adaptively trained long-short-term memory deep neural network was developed and trained using a modest number of seizure data sets from wrist-worn devices. Transfer learning was used to adapt a classifier that was initially trained on intracranial electroencephalography (iEEG) signals to facilitate classification of non-EEG physiological datasets comprising accelerometry, blood volume pulse, skin electrodermal activity, heart rate, and temperature signals. The algorithm's performance was assessed with and without pre-training on iEEG signals and transfer learning. To assess the performance of the seizure detection classifier using long-term ambulatory data, wearable devices were used for multiple months with an implanted neurostimulator capable of recording iEEG signals, which provided independent electrographic seizure detections that were reviewed by a board-certified epileptologist.Main results. For 19 motor seizures from 10 in-hospital patients, the algorithm yielded a mean area under curve (AUC), a sensitivity, and an false alarm rate per day (FAR/day) of 0.98, 0.93, and 2.3, respectively. Additionally, for eight seizures with probable motor semiology from two ambulatory patients, the classifier achieved a mean AUC of 0.97 and an FAR of 2.45 events/day at a sensitivity of 0.9. For all seizure types in the ambulatory setting, the classifier had a mean AUC of 0.82 with a sensitivity of 0.47 and an FAR of 7.2 events/day.Significance. The performance of the algorithm was evaluated using motor and non-motor seizures during in-hospital and ambulatory use. The classifier was able to detect multiple types of motor and non-motor seizures, but performed significantly better on motor seizures.
TL;DR: In this paper, the histopathological effects of Pt within the cochlea following continuous stimulation at a charge density well above the defined safe limits for periods up to 6 months were examined.
Abstract: OBJECTIVE: Established guidelines for safe levels of electrical stimulation for neural prostheses are based on a limited range of the stimulus parameters used clinically. Recent studies have reported particulate platinum (Pt) associated with long-term clinical use of these devices, highlighting the need for more carefully defined safety limits. We previously reported no adverse effects of Pt corrosion products in the cochleae of guinea pigs following 4 weeks of electrical stimulation using charge densities far greater than the published safe limits for cochlear implants. The present study examines the histopathological effects of Pt within the cochlea following continuous stimulation at a charge density well above the defined safe limits for periods up to 6 months. APPROACH: Six cats were bilaterally implanted with Pt electrode arrays and unilaterally stimulated using charge balanced current pulses at a charge density of 267 C/cm2/phase using a tripolar electrode configuration. Electrochemical measurements were made throughout the implant duration and evoked potentials recorded at the outset and on completion of the stimulation program. Cochleae were examined histologically for particulate Pt, tissue response, and auditory nerve survival; electrodes were examined for surface corrosion; and cochlea, brain, kidney, and liver tissue analysed for trace levels of Pt. MAIN RESULTS: Chronic stimulation resulted in both a significant increase in tissue response and particulate Pt within the tissue capsule surrounding the electrode array compared with implanted, unstimulated control cochleae. Importantly, there was no stimulus-induced loss of auditory neurons or increase in evoked potential thresholds. Stimulated electrodes were significantly more corroded compared with unstimulated electrodes. Trace analysis revealed Pt in both stimulated and control cochleae although significantly greater levels were detected within stimulated cochleae. There was no evidence of Pt in brain or liver; however, trace levels of Pt were recorded in the kidneys of two animals. Finally, increased charge storage capacity and charge injection limit reflected the more extensive electrode corrosion associated with stimulated electrodes. SIGNIFICANCE: Long-term electrical stimulation of Pt electrodes at a charge density well above existing safety limits and nearly an order of magnitude higher than levels used clinically, does not adversely affect the auditory neuron population or reduce neural function, despite a stimulus-induced tissue response and the accumulation of Pt corrosion product. The mechanism resulting in Pt within the unstimulated cochlea is unclear, while the level of Pt observed systemically following stimulation at these very high charge densities does not appear to be of clinical significance.
TL;DR: In this article, a deep learning model was used for the classification of subjects with Alzheimer's disease (AD), mild cognitive impairment (MCI) and healthy ageing (HA) using resting-state scalp electroencephalogram (EEG) signals.
Abstract: Objective.This study aimed to produce a novel deep learning (DL) model for the classification of subjects with Alzheimer's disease (AD), mild cognitive impairment (MCI) subjects and healthy ageing (HA) subjects using resting-state scalp electroencephalogram (EEG) signals.Approach.The raw EEG data were pre-processed to remove unwanted artefacts and sources of noise. The data were then processed with the continuous wavelet transform, using the Morse mother wavelet, to create time-frequency graphs with a wavelet coefficient scale range of 0-600. The graphs were combined into tiled topographical maps governed by the 10-20 system orientation for scalp electrodes. The application of this processing pipeline was used on a data set of resting-state EEG samples from age-matched groups of 52 AD subjects (82.3 ± 4.7 years of age), 37 MCI subjects (78.4 ± 5.1 years of age) and 52 HA subjects (79.6 ± 6.0 years of age). This resulted in the formation of a data set of 16197 topographical images. This image data set was then split into training, validation and test images and used as input to an AlexNet DL model. This model was comprised of five hidden convolutional layers and optimised for various parameters such as learning rate, learning rate schedule, optimiser, and batch size.Main results.The performance was assessed by a tenfold cross-validation strategy, which produced an average accuracy result of 98.9 ± 0.4% for the three-class classification of AD vs MCI vs HA. The results showed minimal overfitting and bias between classes, further indicating the strength of the model produced.Significance.These results provide significant improvement for this classification task compared to previous studies in this field and suggest that DL could contribute to the diagnosis of AD from EEG recordings.
TL;DR: A review of the development of flexible and implantable peripheral nerve interfaces can be found in this article, highlighting promising solutions related to materials selection and their associated fabrication methods, and integrated functions.
Abstract: Peripheral nerve interfaces (PNIs) record and/or modulate neural activity of nerves, which are responsible for conducting sensory-motor information to and from the central nervous system, and for regulating the activity of inner organs. PNIs are used both in neuroscience research and in therapeutical applications such as precise closed-loop control of neuroprosthetic limbs, treatment of neuropathic pain and restoration of vital functions (e.g. breathing and bladder management). Implantable interfaces represent an attractive solution to directly access peripheral nerves and provide enhanced selectivity both in recording and in stimulation, compared to their non-invasive counterparts. Nevertheless, the long-term functionality of implantable PNIs is limited by tissue damage, which occurs at the implant-tissue interface, and is thus highly dependent on material properties, biocompatibility and implant design. Current research focuses on the development of mechanically compliant PNIs, which adapt to the anatomy and dynamic movements of nerves in the body thereby limiting foreign body response. In this paper, we review recent progress in the development of flexible and implantable PNIs, highlighting promising solutions related to materials selection and their associated fabrication methods, and integrated functions. We report on the variety of available interface designs (intraneural, extraneural and regenerative) and different modulation techniques (electrical, optical, chemical) emphasizing the main challenges associated with integrating such systems on compliant substrates.
TL;DR: In this article, a sliding window based weighted majority voting algorithm was developed to detect seizure events based on each DL model's classification results, with an accuracy degradation of less than 0.5%.
Abstract: Objective.Implanted devices providing real-time neural activity classification and control are increasingly used to treat neurological disorders, such as epilepsy and Parkinson's disease. Classification performance is critical to identifying brain states appropriate for the therapeutic action (e.g. neural stimulation). However, advanced algorithms that have shown promise in offline studies, in particular deep learning (DL) methods, have not been deployed on resource-restrained neural implants. Here, we designed and optimized three DL models or edge deployment and evaluated their inference performance in a case study of seizure detection.Approach.A deep neural network (DNN), a convolutional neural network (CNN), and a long short-term memory (LSTM) network were designed and trained with TensorFlow to classify ictal, preictal, and interictal phases from the CHB-MIT scalp EEG database. A sliding window based weighted majority voting algorithm was developed to detect seizure events based on each DL model's classification results. After iterative model compression and coefficient quantization, the algorithms were deployed on a general-purpose, off-the-shelf microcontroller for real-time testing. Inference sensitivity, false positive rate (FPR), execution time, memory size, and power consumption were quantified.Main results.For seizure event detection, the sensitivity and FPR for the DNN, CNN, and LSTM models were 87.36%/0.169 h-1, 96.70%/0.102 h-1, and 97.61%/0.071 h-1, respectively. Predicting seizures for early warnings was also feasible. The LSTM model achieved the best overall performance at the expense of the highest power. The DNN model achieved the shortest execution time. The CNN model showed advantages in balanced performance and power with minimum memory requirement. The implemented model compression and quantization achieved a significant saving of power and memory with an accuracy degradation of less than 0.5%.Significance.Inference with embedded DL models achieved performance comparable to many prior implementations that had no time or computational resource limitations. Generic microcontrollers can provide the required memory and computational resources, while model designs can be migrated to application-specific integrated circuits for further optimization and power saving. The results suggest that edge DL inference is a feasible option for future neural implants to improve classification performance and therapeutic outcomes.
TL;DR: The potential of fuzzy entropy for evaluating the temporal variability of the dynamic resting-state brain networks, and the fuzzy entropy is also helpful for uncovering the fluctuating network variability that accounts for the individual decision differences are suggested.
Abstract: Objective.Exploring the temporal variability in spatial topology during the resting state attracts growing interest and becomes increasingly useful to tackle the cognitive process of brain networks. In particular, the temporal brain dynamics during the resting state may be delineated and quantified aligning with cognitive performance, but few studies investigated the temporal variability in the electroencephalogram (EEG) network as well as its relationship with cognitive performance.Approach.In this study, we proposed an EEG-based protocol to measure the nonlinear complexity of the dynamic resting-state network by applying the fuzzy entropy. To further validate its applicability, the fuzzy entropy was applied into simulated and two independent datasets (i.e. decision-making and P300).Main results.The simulation study first proved that compared to the existing methods, this approach could not only exactly capture the pattern dynamics in time series but also overcame the magnitude effect of time series. Concerning the two EEG datasets, the flexible and robust network architectures of the brain cortex at rest were identified and distributed at the bilateral temporal lobe and frontal/occipital lobe, respectively, whose variability metrics were found to accurately classify different groups. Moreover, the temporal variability of resting-state network property was also either positively or negatively related to individual cognitive performance.Significance.This outcome suggested the potential of fuzzy entropy for evaluating the temporal variability of the dynamic resting-state brain networks, and the fuzzy entropy is also helpful for uncovering the fluctuating network variability that accounts for the individual decision differences.
TL;DR: In this article, RNA-sequencing was used to identify the spatiotemporal gene expression patterns in interfacial (within 100 µm) and distal (500 µm from implant) brain tissue around implanted silicon microelectrode arrays.
Abstract: Objective.Intracortical brain interfaces are an ever evolving technology with growing potential for clinical and research applications. The chronic tissue response to these devices traditionally has been characterized by glial scarring, inflammation, oxidative stress, neuronal loss, and blood-brain barrier disruptions. The full complexity of the tissue response to implanted devices is still under investigation.Approach.In this study, we have utilized RNA-sequencing to identify the spatiotemporal gene expression patterns in interfacial (within 100µm) and distal (500µm from implant) brain tissue around implanted silicon microelectrode arrays. Naive, unimplanted tissue served as a control.Main results.The data revealed significant overall differential expression (DE) in contrasts comparing interfacial tissue vs naive (157 DE genes), interfacial vs distal (94 DE genes), and distal vs naive tissues (21 DE genes). Our results captured previously characterized mechanisms of the foreign body response, such as astroglial encapsulation, as well as novel mechanisms which have not yet been characterized in the context of indwelling neurotechnologies. In particular, we have observed perturbations in multiple neuron-associated genes which potentially impact the intrinsic function and structure of neurons at the device interface. In addition to neuron-associated genes, the results presented in this study identified significant DE in genes which are associated with oligodendrocyte, microglia, and astrocyte involvement in the chronic tissue response.Significance. The results of this study increase the fundamental understanding of the complexity of tissue response in the brain and provide an expanded toolkit for future investigation into the bio-integration of implanted electronics with tissues in the central nervous system.
TL;DR: Results indicate that beta projections to the spinal motor neuron pool can be voluntarily controlled partially decoupled from natural muscle contractions and, therefore, they could be valid control signals for implementing effective human motor augmentation platforms.
Abstract: Effective human motor augmentation should rely on biological signals that can be volitionally modulated without compromising natural motor control. We provided human subjects with real-time information on the power of two separate spectral bands of the spiking activity of motor neurons innervating the tibialis anterior muscle: the low-frequency band (<7Hz), which is directly translated into natural force control, and the beta band (13-30Hz), which is outside the dynamics of the neuromuscular system. Subjects could gain control over the powers in these two bands to navigate a cursor towards specific targets in a 2-D space (experiment 1) and to up- and down-modulate beta activity while keeping steady force contractions (experiment 2). Results indicate that beta projections to the spinal motor neuron pool can be voluntarily controlled partially decoupled from natural muscle contractions and, therefore, they could be valid control signals for implementing effective human motor augmentation platforms.
TL;DR: In this article, the authors describe the development, validation, and dissemination of flexible, high-resolution, thin-film (TF) electrodes for recording neural activity in animals and humans.
Abstract: Objective.Brain functions such as perception, motor control, learning, and memory arise from the coordinated activity of neuronal assemblies distributed across multiple brain regions. While major progress has been made in understanding the function of individual neurons, circuit interactions remain poorly understood. A fundamental obstacle to deciphering circuit interactions is the limited availability of research tools to observe and manipulate the activity of large, distributed neuronal populations in humans. Here we describe the development, validation, and dissemination of flexible, high-resolution, thin-film (TF) electrodes for recording neural activity in animals and humans.Approach.We leveraged standard flexible printed-circuit manufacturing processes to build high-resolution TF electrode arrays. We used biocompatible materials to form the substrate (liquid crystal polymer; LCP), metals (Au, PtIr, and Pd), molding (medical-grade silicone), and 3D-printed housing (nylon). We designed a custom, miniaturized, digitizing headstage to reduce the number of cables required to connect to the acquisition system and reduce the distance between the electrodes and the amplifiers. A custom mechanical system enabled the electrodes and headstages to be pre-assembled prior to sterilization, minimizing the setup time required in the operating room. PtIr electrode coatings lowered impedance and enabled stimulation. High-volume, commercial manufacturing enables cost-effective production of LCP-TF electrodes in large quantities.Main Results. Our LCP-TF arrays achieve 25× higher electrode density, 20× higher channel count, and 11× reduced stiffness than conventional clinical electrodes. We validated our LCP-TF electrodes in multiple human intraoperative recording sessions and have disseminated this technology to >10 research groups. Using these arrays, we have observed high-frequency neural activity with sub-millimeter resolution.Significance.Our LCP-TF electrodes will advance human neuroscience research and improve clinical care by enabling broad access to transformative, high-resolution electrode arrays.
TL;DR: The proposed LE-ConvMN method could efficiently estimate the continuous movement of fingers with sEMG, and its performance is significantly superior to two established deep learning methods.
Abstract: Objective Estimation of finger kinematics is an important function of an intuitive human-machine interface, such as gesture recognition. Here, we propose a novel deep learning method, named Long Exposure Convolutional Memory Network (LE-ConvMN), and use it to proportionally estimate finger joint angles through surface electromyographic (sEMG) signals. Approach We use a convolution structure to replace the neuron structure of traditional Long Short-Term Memory (LSTM) networks, and use the long exposure data structure which retains the spatial and temporal information of the electrodes as input. The Ninapro database, which contains continuous finger gestures and corresponding sEMG signals was used to verify the efficiency of the proposed deep learning method. The proposed method was compared with LSTM and Sparse Pseudo-input Gaussian Process (SPGP) on this database to predict the 10 main joint angles on the hand based on sEMG. The correlation coefficient (CC) was evaluated using the three methods on eight healthy subjects, and all the methods adopted the root mean square (RMS) features. Main results The experimental results showed that the average CC, RMSE, NRMSE of the proposed LE-ConvMN method (0.82±0.03,11.54±1.89,0.12±0.013) was significantly higher than SPGP (0.65±0.05, p＜0.001; 15.51±2.82, p＜0.001; 0.16±0.01, p＜0.001) and LSTM (0.64±0.06, p＜0.001; 14.77±3.21, p＜0.001; 0.15±0.02, p=＜0.001). Furthermore, the proposed real-time-estimation method has a computation cost of only approximately 82 ms to output one state of ten joints (average value of 10 tests on TitanV GPU). Significance The proposed LE-ConvMN method could efficiently estimate the continuous movement of fingers with sEMG, and its performance is significantly superior to two established deep learning methods.
TL;DR: Zhang et al. as discussed by the authors developed a new deep learning approach, which employs a long short-term memory (LSTM) network architecture and an auxiliary classifier generative adversarial network (AC-GAN), to train on both expert-annotated and augmented spike events from intracranial electroencephalography (iEEG) recordings of epilepsy patients.
Abstract: Objective Automatic detection of interictal epileptiform discharges (IEDs, short as ``spikes'') from an epileptic brain can help predict seizure recurrence and support the diagnosis of epilepsy. Developing fast, reliable and robust detection methods for IEDs based on scalp or intracortical EEG may facilitate online seizure monitoring and closed-loop neurostimulation. Approach We developed a new deep learning approach, which employs a long short-term memory (LSTM) network architecture (``IEDnet'') and an auxiliary classifier generative adversarial network (AC-GAN), to train on both expert-annotated and augmented spike events from intracranial electroencephalography (iEEG) recordings of epilepsy patients. We validated our IEDnet with two real-world iEEG datasets, and compared IEDnet with the support vector machine (SVM) and random forest (RF) classifiers on their detection performances. Main results IEDnet achieved excellent cross-validated detection performances in terms of both sensitivity and specificity, and outperformed SVM and RF. Synthetic spike samples augmented by AC-GAN further improved the detection performance. In addition, the performance of IEDnet was robust with respect to the sampling frequency and noise. Furthermore, we also demonstrated the cross-institutional generalization ability of IEDnet while testing between two datasets. Significance IEDnet achieves excellent detection performances in identifying interictal spikes. AC-GAN can produce augmented iEEG samples to improve supervised deep learning.
TL;DR: In this paper, the authors used CNN-based time series classification (TSC) methods to classify left and right hand motor imagery tasks, reaching 98.6% accuracy on overall person, single person and overall person with single-channel classification results.
Abstract: Objective. Development of a brain-computer interface (BCI) requires classification of brain neural activities to different states. Functional near-infrared spectroscopy (fNIRS) can measure the brain activities and has great potential for BCI. In recent years, a large number of classification algorithms have been proposed, in which deep learning methods, especially convolutional neural network (CNN) methods are successful. fNIRS signal has typical time series properties, we combined fNIRS data and kinds of CNN-based time series classification (TSC) methods to classify BCI task.Approach. In this study, participants were recruited for a left and right hand motor imagery experiment and the cerebral neural activities were recorded by fNIRS equipment (FOIRE-3000). TSC methods are used to distinguish the brain activities when imagining the left or right hand. We have tested the overall person, single person and overall person with single-channel classification results, and these methods achieved excellent classification results. We also compared the CNN-based TSC methods with traditional classification methods such as support vector machine.Main results. Experiments showed that the CNN-based methods have significant advantages in classification accuracy: the CNN-based methods have achieved remarkable results in the classification of left-handed and right-handed imagination tasks, reaching 98.6% accuracy on overall person, 100% accuracy on single person, and in the single-channel classification an accuracy of 80.1% has been achieved with the best-performing channel.Significance. These results suggest that using the CNN-based TSC methods can significantly improve the BCI performance and also lay the foundation for the miniaturization and portability of training rehabilitation equipment.