Showing papers by "Paul Sajda published in 2016"

Brain dynamics of post-task resting state are influenced by expertise: Insights from baseball players.

Abstract: Post-task resting state dynamics can be viewed as a task-driven state where behavioral performance is improved through endogenous, non-explicit learning. Tasks that have intrinsic value for individuals are hypothesized to produce post-task resting state dynamics that promote learning. We measured simultaneous fMRI/EEG and DTI in Division-1 collegiate baseball players and compared to a group of controls, examining differences in both functional and structural connectivity. Participants performed a surrogate baseball pitch Go/No-Go task before a resting state scan, and we compared post-task resting state connectivity using a seed-based analysis from the supplementary motor area (SMA), an area whose activity discriminated players and controls in our previous results using this task. Although both groups were equally trained on the task, the experts showed differential activity in their post-task resting state consistent with motor learning. Specifically, we found (1) differences in bilateral SMA-L Insula functional connectivity between experts and controls that may reflect group differences in motor learning, (2) differences in BOLD-alpha oscillation correlations between groups suggests variability in modulatory attention in the post-task state, and (3) group differences between BOLD-beta oscillations that may indicate cognitive processing of motor inhibition. Structural connectivity analysis identified group differences in portions of the functionally derived network, suggesting that functional differences may also partially arise from variability in the underlying white matter pathways. Generally, we find that brain dynamics in the post-task resting state differ as a function of subject expertise and potentially result from differences in both functional and structural connectivity. Hum Brain Mapp 37:4454-4471, 2016. © 2016 The Authors Human Brain Mapping Published by Wiley Periodicals, Inc.

Unsupervised adaptive transfer learning for Steady-State Visual Evoked Potential brain-computer interfaces

Abstract: Recent advances in signal processing for the detection of Steady-State Visual Evoked Potentials (SSVEPs) have moved away from traditionally calibrationless methods, such as canonical correlation analysis, and towards algorithms that require substantial training data. In general, this has improved detection rates, but SSVEP-based brain-computer interfaces (BCIs) now suffer from the requirement of costly calibration sessions. Here, we address this issue by applying transfer learning techniques to SSVEP detection. Our novel Adaptive-C3A method incorporates an unsupervised adaptation algorithm that requires no calibration data. Our approach learns SSVEP templates for the target user and provides robust class separation in feature space leading to increased classification accuracy. Our method achieves significant improvements in performance over a standard CCA method as well as a transfer variant of the state-of-the art Combined-CCA method for calibrationless SSVEP detection.

Cortically Coupled Computing: A New Paradigm for Synergistic Human-Machine Interaction

Abstract: Unlike traditional brain-computer interfaces that use brain signals for direct control of computers and robotics, a cortically coupled computer system opportunistically senses the brain state, capturing a user's implicit or explicit computation, and then communicates this information to a traditional computer system via a neural interface.

Neural mechanisms underlying catastrophic failure in human-machine interaction during aerial navigation.

Abstract: Objective. We investigated the neural correlates of workload buildup in a fine visuomotor task called the boundary avoidance task (BAT). The BAT has been known to induce naturally occurring failures of human–machine coupling in high performance aircraft that can potentially lead to a crash—these failures are termed pilot induced oscillations (PIOs). Approach. We recorded EEG and pupillometry data from human subjects engaged in a flight BAT simulated within a virtual 3D environment. Main results. We find that workload buildup in a BAT can be successfully decoded from oscillatory features in the electroencephalogram (EEG). Information in delta, theta, alpha, beta, and gamma spectral bands of the EEG all contribute to successful decoding, however gamma band activity with a lateralized somatosensory topography has the highest contribution, while theta band activity with a fronto-central topography has the most robust contribution in terms of real-world usability. We show that the output of the spectral decoder can be used to predict PIO susceptibility. We also find that workload buildup in the task induces pupil dilation, the magnitude of which is significantly correlated with the magnitude of the decoded EEG signals. These results suggest that PIOs may result from the dysregulation of cortical networks such as the locus coeruleus (LC)—anterior cingulate cortex (ACC) circuit. Significance. Our findings may generalize to similar control failures in other cases of tight man-machine coupling where gains and latencies in the control system must be inferred and compensated for by the human operators. A closed-loop intervention using neurophysiological decoding of workload buildup that targets the LC-ACC circuit may positively impact operator performance in such situations.

Towards Serious Games for Improved BCI

Abstract: : Brain-computer interface (BCI) technologies, or technologies that use online brain signal processing, have a great promise to improve human interactions with computers, their environment, and even other humans. Despite this promise, there are no current serious BCI technologies in widespread use, due to the lack of robustness in BCI technologies. The key neural aspect of this lack of robustness is human variability, which has two main components: (1) individual differences in neural signals and (2) intraindividual variability over time. In order to develop widespread BCI technologies, it will be necessary to address this lack of robustness. However, it is currently unknown how neural variability affects BCI performance. To accomplish these goals, it is essential to obtain data from large numbers of individuals using BCI technologies over considerable lengths of time. One promising method for this is through the use of BCI technologies embedded into games with a purpose (GWAP). GWAP are a game-based form of crowdsourcing which players choose to play for enjoyment and during which the player performs key tasks which cannot be automated but that are required to solve research questions. By embedding BCI paradigms in GWAP and recording neural and behavioral data, it should be possible to much more clearly understand the differences in neural signals between individuals and across different time scales, enabling the development of novel and increasingly robust adaptive BCI algorithms.

Imaging decision-related neural cascades in the human brain

Abstract: Perceptual decisions depend on coordinated patterns of neural activity cascading across the brain, running in time from stimulus to response and in space from primary sensory regions to the frontal lobe. Measuring this cascade and how it flows through the brain is key to developing an understanding of how our brains function. However observing, let alone understanding, this cascade, particularly in humans, is challenging. Here, we report a significant methodological advance allowing this observation in humans at unprecedented spatiotemporal resolution. We use a novel encoding model to link simultaneously measured electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) signals to infer the high-resolution spatiotemporal brain dynamics taking place during rapid visual perceptual decision-making. After demonstrating the methodology replicates past results, we show that it uncovers a previously unobserved sequential reactivation of a substantial fraction of the pre-response network whose magnitude correlates with decision confidence. Our results illustrate that a temporally coordinated and spatially distributed neural cascade underlies perceptual decision-making, with our methodology illuminating complex brain dynamics that would otherwise be unobservable using conventional fMRI or EEG separately. We expect this methodology to be useful in observing brain dynamics in a wide range of other mental processes.

Closed-loop regulation of user state during a boundary avoidance task

Abstract: Pilot induced oscillations (PIOs) are potentially catastrophic events that occur during flight when pilots attempt to control an aircraft close to a performance or physical boundary. PIO-like behavior is typically observed in boundary avoidance tasks (BAT), which simulate tight performance or physical boundaries and induce high cognitive workload. Our previous research linked the occurrence of PIO-like behavior to network level activity in the brain, where higher states of arousal reduce the flexibility of decision making networks such that less environmental information was incorporated to dynamically adjust action. This led us to hypothesize that down regulating arousal via closed-loop audio feedback of a user state could improve piloting performance by enabling increased decision flexibility. Here we show our initial results testing this hypothesis, where we use a hybrid brain computer interface (hBCI) to dynamically provide feedback to a “pilot” that facilitates their ability to reduce their state of arousal. We conduct a systematic comparison relative to control and sham conditions and test to see if this feedback increases the time a “pilot” can fly before a catastrophic PIO. We find that hBCI feedback, which includes central nervous system components consistent with theta activity in the anterior cingulate cortex (ACC), enables prolonged flight relative to closed-loop control and sham feedback. We also find that this feedback induces changes in pupil diameter which are absent in openloop conditions and closed-loop conditions when feedback is not veridical. Pupil diameter has been reported as a surrogate measure of activity in the locus coeruleus-norepinephrine (LC-NE) system which is also linked to a circuit that includes the ACC. We conclude that the feedback we induce with our hBCI provides preliminary evidence that self-regulation of LC-NE/ACC is possible and can be used to dynamically increase decision flexibility when under high cognitive workload.

Predicting decision accuracy and certainty in complex brain-machine interactions

Abstract: A promising application of brain machine interfaces (BMIs) is predicting user cognitive state, particularly in complex and demanding scenarios, so that automation can dynamically and adaptively adjust task parameters to optimize joint human-machine performance. In this paper we analyze neural, physiological and behavioral data recorded during a complex two-person “crew station” task and investigate whether these measures provide information for inferring user decision state. Specifically, we investigate how measures of EEG, pupil dilation, heart rate and response time, can be fused to infer decision confidence and accuracy in two side-tasks occurring throughout a three hour experimental session. One side-task is an auditory task, the other a visual task, both occurring within the context of the crew station scenario (auditory alert and a visual satellite map N-back task). We find that the best prediction performance always fuses EEG and pupil dilation measures, with results yielding between 70%–75% accuracy with respect to whether the subject(s) will skip making the decision (i.e. have high uncertainty) or whether he/she makes an error. Interestingly, the results suggest a possible mechanistic explanation for the utility of the fused measures, specifically the interaction between the locus coeruleus (LC), whose activity is linked to arousal state and can be inferred from pupil dilation, and the anterior cingulate (ACC), which has been linked to decision formation and monitoring and whose activity is typically measured via EEG. In general, our results demonstrate the potential in using fused neuro/physio measures to infer and track human operator decision uncertainty during demanding complex tasks, possibly enabling BMIs to eventually be employed as “cognitive orthotics” for improving man-machine interaction and performance.

Multidimensional stimulus-response correlation reveals supramodal neural responses to naturalistic stimuli

Abstract: In neuroscience, stimulus-response relationships have traditionally been analyzed using either encoding or decoding models. Here we combined both techniques by decomposing neural activity into multiple components, each representing a portion of the stimulus. We tested this hybrid approach on encephalographic responses to auditory and audiovisual narratives identically experienced across subjects, as well as uniquely experienced video game play. The highest stimulus-response correlations (SRC) were detected for dynamic visual features. During narratives both auditory and visual SRC were modulated by attention and tracked correlations between subjects. During video game play, SRC was modulated by task difficulty and attentional state. Importantly, the strongest component extracted for visual and auditory features had nearly identical spatial distributions, suggesting that the predominant encephalographic response to naturalistic stimuli is supramodal. The variety of novel findings demonstrates the utility of measuring multidimensional stimulus-response correlations.

Title: Imaging decision-related neural cascades in the human brain 1 2

Abstract: 20 Perceptual decisions depend on coordinated patterns of neural activity cascading across 21 the brain, running in time from stimulus to response and in space from primary sensory 22 regions to the frontal lobe. Measuring this cascade and how it flows through the brain is 23 key to developing an understanding of how our brains function. However observing, let 24 alone understanding, this cascade, particularly in humans, is challenging. Here, we report 25 a significant methodological advance allowing this observation in humans at 26 unprecedented spatiotemporal resolution. We use a novel encoding model to link 27 simultaneously measured electroencephalography (EEG) and functional magnetic 28 resonance imaging (fMRI) signals to infer the high-resolution spatiotemporal brain 29 dynamics taking place during rapid visual perceptual decision-making. After 30 demonstrating the methodology replicates past results, we show that it uncovers a 31 previously unobserved sequential reactivation of a substantial fraction of the pre-response 32 network whose magnitude correlates with decision confidence. Our results illustrate that 33 a temporally coordinated and spatially distributed neural cascade underlies perceptual 34 decision-making, with our methodology illuminating complex brain dynamics that would 35 otherwise be unobservable using conventional fMRI or EEG separately. We expect this 36 methodology to be useful in observing brain dynamics in a wide range of other mental 37 processes. 38

Neural mechanisms underlying catastrophic failure in human-machine interaction during aerial navigation

Abstract: Objective. We investigated the neural correlates of workload buildup in a fine visuomotor task called the boundary avoidance task (BAT). The BAT has been known to induce naturally occurring failures of human-machine coupling in high performance aircraft that can potentially lead to a crash; these failures are termed pilot induced oscillations (PIOs). Approach. We recorded EEG and pupillometry data from human subjects engaged in a flight BAT simulated within a virtual 3D environment. Main results. We find that workload buildup in a BAT can be successfully decoded from oscillatory features in the electroencephalogram (EEG). Information in delta, theta, alpha, beta, and gamma spectral bands of the EEG all contribute to successful decoding, however gamma band activity with a lateralized somatosensory topography has the highest contribution, while theta band activity with a frontocentral topography has the most robust contribution in terms of real world usability. We show that the output of the spectral decoder can be used to predict PIO susceptibility. We also find that workload buildup in the task induces pupil dilation, the magnitude of which is significantly correlated with the magnitude of the decoded EEG signals. These results suggest that PIOs may result from the dysregulation of cortical networks such as the locus coeruleus (LC) anterior cingulate cortex (ACC) circuit. Significance. Our findings may generalize to similar control failures in other cases of tight man machine coupling where gains and latencies in the control system must be inferred and compensated for by the human operators. A closed-loop intervention using neurophysiological decoding of workload buildup that targets the LC ACC circuit may positively impact operator performance in such situations.