Showing papers in "Journal of Cognitive Neuroscience in 2020"

A Tale of Two Positivities and the N400: Distinct Neural Signatures Are Evoked by Confirmed and Violated Predictions at Different Levels of Representation.

Abstract: It has been proposed that hierarchical prediction is a fundamental computational principle underlying neurocognitive processing. Here, we ask whether the brain engages distinct neurocognitive mechanisms in response to inputs that fulfill versus violate strong predictions at different levels of representation during language comprehension. Participants read three-sentence scenarios in which the third sentence constrained for a broad event structure, for example, {Agent caution animate-Patient}. High constraint contexts additionally constrained for a specific event/lexical item, for example, a two-sentence context about a beach, lifeguards, and sharks constrained for the event, {Lifeguards cautioned Swimmers}, and the specific lexical item swimmers. Low constraint contexts did not constrain for any specific event/lexical item. We measured ERPs on critical nouns that fulfilled and/or violated each of these constraints. We found clear, dissociable effects to fulfilled semantic predictions (a reduced N400), to event/lexical prediction violations (an increased late frontal positivity), and to event structure/animacy prediction violations (an increased late posterior positivity/P600). We argue that the late frontal positivity reflects a large change in activity associated with successfully updating the comprehender's current situation model with new unpredicted information. We suggest that the late posterior positivity/P600 is triggered when the comprehender detects a conflict between the input and her model of the communicator and communicative environment. This leads to an initial failure to incorporate the unpredicted input into the situation model, which may be followed by second-pass attempts to make sense of the discourse through reanalysis, repair, or reinterpretation. Together, these findings provide strong evidence that confirmed and violated predictions at different levels of representation manifest as distinct spatiotemporal neural signatures.

Effects of Maternal Singing Style on Mother-Infant Arousal and Behavior.

Abstract: Mothers around the world sing to infants, presumably to regulate their mood and arousal. Lullabies and playsongs differ stylistically and have distinctive goals. Mothers sing lullabies to soothe an...

Cortical Tracking of Surprisal during Continuous Speech Comprehension

Abstract: Speech comprehension requires rapid online processing of a continuous acoustic signal to extract structure and meaning. Previous studies on sentence comprehension have found neural correlates of the predictability of a word given its context, as well as of the precision of such a prediction. However, they have focused on single sentences and on particular words in those sentences. Moreover, they compared neural responses to words with low and high predictability, as well as with low and high precision. However, in speech comprehension, a listener hears many successive words whose predictability and precision vary over a large range. Here, we show that cortical activity in different frequency bands tracks word surprisal in continuous natural speech and that this tracking is modulated by precision. We obtain these results through quantifying surprisal and precision from naturalistic speech using a deep neural network and through relating these speech features to EEG responses of human volunteers acquired during auditory story comprehension. We find significant cortical tracking of surprisal at low frequencies, including the delta band as well as in the higher frequency beta and gamma bands, and observe that the tracking is modulated by the precision. Our results pave the way to further investigate the neurobiology of natural speech comprehension.

A Compositional Neural Architecture for Language

Abstract: Hierarchical structure and compositionality imbue human language with unparalleled expressive power and set it apart from other perception-action systems. However, neither formal nor neurobiological models account for how these defining computational properties might arise in a physiological system. I attempt to reconcile hierarchy and compositionality with principles from cell assembly computation in neuroscience; the result is an emerging theory of how the brain could convert distributed perceptual representations into hierarchical structures across multiple timescales while representing interpretable incremental stages of (de)compositional meaning. The model's architecture-a multidimensional coordinate system based on neurophysiological models of sensory processing-proposes that a manifold of neural trajectories encodes sensory, motor, and abstract linguistic states. Gain modulation, including inhibition, tunes the path in the manifold in accordance with behavior and is how latent structure is inferred. As a consequence, predictive information about upcoming sensory input during production and comprehension is available without a separate operation. The proposed processing mechanism is synthesized from current models of neural entrainment to speech, concepts from systems neuroscience and category theory, and a symbolic-connectionist computational model that uses time and rhythm to structure information. I build on evidence from cognitive neuroscience and computational modeling that suggests a formal and mechanistic alignment between structure building and neural oscillations, and moves toward unifying basic insights from linguistics and psycholinguistics with the currency of neural computation.

Distributed Patterns of Functional Connectivity Predict Working Memory Performance in Novel Healthy and Memory-impaired Individuals

Abstract: Individual differences in working memory relate to performance differences in general cognitive ability. The neural bases of such individual differences, however, remain poorly understood. Here, us...

Linked Sources of Neural Noise Contribute to Age-related Cognitive Decline.

Abstract: Healthy aging is associated with a multitude of structural changes in the brain. These physical age-related changes are accompanied by increased variability in neural activity of all kinds, and thi...

Fixation-related Brain Potentials during Semantic Integration of Object-Scene Information.

Abstract: In vision science, a particularly controversial topic is whether and how quickly the semantic information about objects is available outside foveal vision. Here, we aimed at contributing to this debate by coregistering eye movements and EEG while participants viewed photographs of indoor scenes that contained a semantically consistent or inconsistent target object. Linear deconvolution modeling was used to analyze the ERPs evoked by scene onset as well as the fixation-related potentials (FRPs) elicited by the fixation on the target object (t) and by the preceding fixation (t - 1). Object-scene consistency did not influence the probability of immediate target fixation or the ERP evoked by scene onset, which suggests that object-scene semantics was not accessed immediately. However, during the subsequent scene exploration, inconsistent objects were prioritized over consistent objects in extrafoveal vision (i.e., looked at earlier) and were more effortful to process in foveal vision (i.e., looked at longer). In FRPs, we demonstrate a fixation-related N300/N400 effect, whereby inconsistent objects elicit a larger frontocentral negativity than consistent objects. In line with the behavioral findings, this effect was already seen in FRPs aligned to the pretarget fixation t - 1 and persisted throughout fixation t, indicating that the extraction of object semantics can already begin in extrafoveal vision. Taken together, the results emphasize the usefulness of combined EEG/eye movement recordings for understanding the mechanisms of object-scene integration during natural viewing.

Capture by Context Elements, Not Attentional Suppression of Distractors, Explains the PD with Small Search Displays

Abstract: Top–down control of attention allows us to resist attentional capture by salient stimuli that are irrelevant to our current goals. Recently, it was proposed that attentional suppression of salient ...

Preservation and Changes in Oscillatory Dynamics across the Cortical Hierarchy.

Abstract: Theta (2-8 Hz), alpha (8-12 Hz), beta (12-35 Hz), and gamma (>35 Hz) rhythms are ubiquitous in the cortex However, there is little understanding of whether they have similar properties and functions in different cortical areas because they have rarely been compared across them We record neuronal spikes and local field potentials simultaneously at several levels of the cortical hierarchy in monkeys Theta, alpha, beta, and gamma oscillations had similar relationships to spiking activity in visual, parietal, and prefrontal cortices However, the frequencies in all bands increased up the cortical hierarchy These results suggest that these rhythms have similar inhibitory and excitatory functions across the cortex We discuss how the increase in frequencies up the cortical hierarchy may help sculpt cortical flow and processing

The Common Rhythm of Action and Perception.

Abstract: Research in the last decade has undermined the idea of perception as a continuous process, providing strong empirical support for its rhythmic modulation. More recently, it has been revealed that the ongoing motor processes influence the rhythmic sampling of sensory information. In this review, we will focus on a growing body of evidence suggesting that oscillation-based mechanisms may structure the dynamic interplay between the motor and sensory system and provide a unified temporal frame for their effective coordination. We will describe neurophysiological data, primarily collected in animals, showing phase-locking of neuronal oscillations to the onset of (eye) movements. These data are complemented by novel evidence in humans, which demonstrate the behavioral relevance of these oscillatory modulations and their domain-general nature. Finally, we will discuss the possible implications of these modulations for action-perception coupling mechanisms.

Failed Suppression of Salient Stimuli Precedes Behavioral Errors.

Abstract: Our visual system is constantly confronted with more information than it can process. To deal with the limited capacity, attention allows us to enhance relevant information and suppress irrelevant information. Particularly, the suppression of salient irrelevant stimuli has shown to be important as it prevents attention to be captured and thus attentional resources to be wasted. This study aimed at directly connecting failures to suppress distraction with a neural marker of suppression, the distractor positivity (Pd). We measured participants' EEG signal while they performed a visual search task in which they had to report a digit inside a shape target while ignoring distractors, one of which could be a salient color singleton. Reports of target digits served as a behavioral index of enhancement, and reports of color distractor digits served as a behavioral index of failed suppression, each measured against reports of neutral distractor digits serving as a baseline. Participants reported the target identity more often than any distractor identity. The singleton identity was reported least often, suggesting suppression of the singleton below baseline. Suppression of salient stimuli was absent in the beginning and then increased throughout the experiment. When the singleton identity was reported, the Pd was observed in a later time window, suggesting that behavioral errors were preceded by failed suppression. Our results provide evidence for the signal suppression hypothesis that states salient items have to be actively suppressed to avoid attentional capture. Our results also provide direct evidence that the Pd is reflecting such active suppression.

Spatial Attention and Temporal Expectation Exert Differential Effects on Visual and Auditory Discrimination

Abstract: Anticipation of an impending stimulus shapes the state of the sensory systems, optimizing neural and behavioral responses. Here, we studied the role of brain oscillations in mediating spatial and temporal anticipations. Because spatial attention and temporal expectation are often associated with visual and auditory processing, respectively, we directly contrasted the visual and auditory modalities and asked whether these anticipatory mechanisms are similar in both domains. We recorded the magnetoencephalogram in healthy human participants performing an auditory and visual target discrimination task, in which cross-modal cues provided both temporal and spatial information with regard to upcoming stimulus presentation. Motivated by prior findings, we were specifically interested in delta (1-3 Hz) and alpha (8-13 Hz) band oscillatory state in anticipation of target presentation and their impact on task performance. Our findings support the view that spatial attention has a stronger effect in the visual domain, whereas temporal expectation effects are more prominent in the auditory domain. For the spatial attention manipulation, we found a typical pattern of alpha lateralization in the visual system, which correlated with response speed. Providing a rhythmic temporal cue led to increased postcue synchronization of low-frequency rhythms, although this effect was more broadband in nature, suggesting a general phase reset rather than frequency-specific neural entrainment. In addition, we observed delta-band synchronization with a frontal topography, which correlated with performance, especially in the auditory task. Combined, these findings suggest that spatial and temporal anticipations operate via a top-down modulation of the power and phase of low-frequency oscillations, respectively.

Individual-subject Functional Localization Increases Univariate Activation but Not Multivariate Pattern Discriminability in the "Multiple-demand" Frontoparietal Network.

Abstract: The frontoparietal "multiple-demand" (MD) control network plays a key role in goal-directed behavior. Recent developments of multivoxel pattern analysis (MVPA) for fMRI data allow for more fine-grained investigations into the functionality and properties of brain systems. In particular, MVPA in the MD network was used to gain better understanding of control processes such as attentional effects, adaptive coding, and representation of multiple task-relevant features, but overall low decoding levels have limited its use for this network. A common practice of applying MVPA is by investigating pattern discriminability within a ROI using a template mask, thus ensuring that the same brain areas are studied in all participants. This approach offers high sensitivity but does not take into account differences between individuals in the spatial organization of brain regions. An alternative approach uses independent localizer data for each subject to select the most responsive voxels and define individual ROIs within the boundaries of a group template. Such an approach allows for a refined and targeted localization based on the unique pattern of activity of individual subjects while ensuring that functionally similar brain regions are studied for all subjects. In the current study, we tested whether using individual ROIs leads to changes in decodability of task-related neural representations as well as univariate activity across the MD network compared with when using a group template. We used three localizer tasks to separately define subject-specific ROIs: spatial working memory, verbal working memory, and a Stroop task. We then systematically assessed univariate and multivariate results in a separate rule-based criterion task. All the localizer tasks robustly recruited the MD network and evoked highly reliable activity patterns in individual subjects. Consistent with previous studies, we found a clear benefit of the subject-specific ROIs for univariate results from the criterion task, with increased activity in the individual ROIs based on the localizers' data, compared with the activity observed when using the group template. In contrast, there was no benefit of the subject-specific ROIs for the multivariate results in the form of increased discriminability, as well as no cost of reduced discriminability. Both univariate and multivariate results were similar in the subject-specific ROIs defined by each of the three localizers. Our results provide important empirical evidence for researchers in the field of cognitive control for the use of individual ROIs in the frontoparietal network for both univariate and multivariate analysis of fMRI data and serve as another step toward standardization and increased comparability across studies.

Getting to Know Someone: Familiarity, Person Recognition, and Identification in the Human Brain

Abstract: In our everyday life, we continuously get to know people, dominantly through their faces. Several neuroscientific experiments showed that familiarization changes the behavioral processing and underlying neural representation of faces of others. Here, we propose a model of the process of how we actually get to know someone. First, the purely visual familiarization of unfamiliar faces occurs. Second, the accumulation of associated, nonsensory information refines person representation, and finally, one reaches a stage where the effortless identification of very well-known persons occurs. We offer here an overview of neuroimaging studies, first evaluating how and in what ways the processing of unfamiliar and familiar faces differs and, second, by analyzing the fMRI adaptation and multivariate pattern analysis results we estimate where identity-specific representation is found in the brain. The available neuroimaging data suggest that different aspects of the information emerge gradually as one gets more and more familiar with a person within the same network. We propose a novel model of familiarity and identity processing, where the differential activation of long-term memory and emotion processing areas is essential for correct identification.

Zooming In and Out on One's Life: Autobiographical Representations at Multiple Time Scales.

Abstract: The ability to decouple from the present environment and explore other times is a central feature of the human mind. Research in cognitive psychology and neuroscience has shown that the personal pa...

Sequence Memory in the Hippocampal-Entorhinal Region.

Abstract: Episodic memories are constructed from sequences of events. When recalling such a memory, we not only recall individual events, but we also retrieve information about how the sequence of events unf...

Individual Alpha Frequency Predicts Perceived Visuotactile Simultaneity

Abstract: Temporal encoding is a key feature in multisensory processing that leads to the integration versus segregation of perceived events over time. Whether or not two events presented at different offsets are perceived as simultaneous varies widely across the general population. Such tolerance to temporal delays is known as the temporal binding window (TBW). It has been recently suggested that individual oscillatory alpha frequency (IAF) peak may represent the electrophysiological correlate of TBW, with IAF also showing a wide variability in the general population (8-12 Hz). In our work, we directly tested this hypothesis by measuring each individual's TBW during a visuotactile simultaneity judgment task while concurrently recording their electrophysiological activity. We found that the individual's TBW significantly correlated with their left parietal IAF, such that faster IAF accounted for narrower TBW. Furthermore, we found that higher prestimulus alpha power measured over the same left parietal regions accounted for more veridical responses of non-simultaneity, which may be explained either by accuracy in perceptual simultaneity or, alternatively, in line with recent proposals by a shift in response bias from more conservative (high alpha power) to more liberal (low alpha power). We propose that the length of an alpha cycle constrains the temporal resolution within which perceptual processes take place.

Content-based Dissociation of Hippocampal Involvement in Prediction.

Abstract: Recent work suggests that a key function of the hippocampus is to predict the future. This is thought to depend on its ability to bind inputs over time and space and to retrieve upcoming or missing inputs based on partial cues. In line with this, previous research has revealed prediction-related signals in the hippocampus for complex visual objects, such as fractals and abstract shapes. Implicit in such accounts is that these computations in the hippocampus reflect domain-general processes that apply across different types and modalities of stimuli. An alternative is that the hippocampus plays a more domain-specific role in predictive processing, with the type of stimuli being predicted determining its involvement. To investigate this, we compared hippocampal responses to auditory cues predicting abstract shapes (Experiment 1) versus oriented gratings (Experiment 2). We measured brain activity in male and female human participants using high-resolution fMRI, in combination with inverted encoding models to reconstruct shape and orientation information. Our results revealed that expectations about shape and orientation evoked distinct representations in the hippocampus. For complex shapes, the hippocampus represented which shape was expected, potentially serving as a source of top-down predictions. In contrast, for simple gratings, the hippocampus represented only unexpected orientations, more reminiscent of a prediction error. We discuss several potential explanations for this content-based dissociation in hippocampal function, concluding that the computational role of the hippocampus in predictive processing may depend on the nature and complexity of stimuli.

A Revised Computational Neuroanatomy for Motor Control.

Abstract: We discuss a new framework for understanding the structure of motor control. Our approach integrates existing models of motor control with the reality of hierarchical cortical processing and the parallel segregated loops that characterize cortical-subcortical connections. We also incorporate the recent claim that cortex functions via predictive representation and optimal information utilization. Our framework assumes that each cortical area engaged in motor control generates a predictive model of a different aspect of motor behavior. In maintaining these predictive models, each area interacts with a different part of the cerebellum and BG. These subcortical areas are thus engaged in domain-appropriate system identification and optimization. This refocuses the question of division of function among different cortical areas. What are the different aspects of motor behavior that are predictively modeled? We suggest that one fundamental division is between modeling of task and body whereas another is the model of state and action. Thus, we propose that the posterior parietal cortex, somatosensory cortex, premotor cortex, and motor cortex represent task state, body state, task action, and body action, respectively. In the second part of this review, we demonstrate how this division of labor can better account for many recent findings of movement encoding, especially in the premotor and posterior parietal cortices.

Perception of Rhythmic Speech Is Modulated by Focal Bilateral Transcranial Alternating Current Stimulation.

Abstract: Several recent studies have used transcranial alternating current stimulation (tACS) to demonstrate a causal role of neural oscillatory activity in speech processing. In particular, it has been sho...

Beat-based and Memory-based Temporal Expectations in Rhythm: Similar Perceptual Effects, Different Underlying Mechanisms.

Abstract: Predicting the timing of incoming information allows the brain to optimize information processing in dynamic environments. Behaviorally, temporal expectations have been shown to facilitate processing of events at expected time points, such as sounds that coincide with the beat in musical rhythm. Yet, temporal expectations can develop based on different forms of structure in the environment, not just the regularity afforded by a musical beat. Little is still known about how different types of temporal expectations are neurally implemented and affect performance. Here, we orthogonally manipulated the periodicity and predictability of rhythmic sequences to examine the mechanisms underlying beat-based and memory-based temporal expectations, respectively. Behaviorally and using EEG, we looked at the effects of beat-based and memory-based expectations on auditory processing when rhythms were task-relevant or task-irrelevant. At expected time points, both beat-based and memory-based expectations facilitated target detection and led to attenuation of P1 and N1 responses, even when expectations were task-irrelevant (unattended). For beat-based expectations, we additionally found reduced target detection and enhanced N1 responses for events at unexpected time points (e.g., off-beat), regardless of the presence of memory-based expectations or task relevance. This latter finding supports the notion that periodicity selectively induces rhythmic fluctuations in neural excitability and furthermore indicates that, although beat-based and memory-based expectations may similarly affect auditory processing of expected events, their underlying neural mechanisms may be different.

Diverse deep neural networks all predict human IT well, after training and fitting

Abstract: Deep neural networks (DNNs) trained on object recognition provide the best current models of high-level visual cortex. What remains unclear is how strongly experimental choices, such as network architecture, training, and fitting to brain data, contribute to the observed similarities. Here, we compare a diverse set of nine DNN architectures on their ability to explain the representational geometry of 62 object images in human inferior temporal (hIT) cortex, as measured with fMRI. We compare untrained networks to their task-trained counterparts and assess the effect of cross-validated fitting to hIT, by taking a weighted combination of the principal components of features within each layer and, subsequently, a weighted combination of layers. For each combination of training and fitting, we test all models for their correlation with the hIT representational dissimilarity matrix, using independent images and subjects. Trained models outperform untrained models (accounting for 57% more of the explainable variance), suggesting that structured visual features are important for explaining hIT. Model fitting further improves the alignment of DNN and hIT representations (by 124%), suggesting that the relative prevalence of different features in hIT does not readily emerge from the Imagenet object-recognition task used to train the networks. The same models can also explain the disparate representations in primary visual cortex (V1), where stronger weights are given to earlier layers. In each region, all architectures achieved equivalently high performance once trained and fitted. The models' shared properties-deep feedforward hierarchies of spatially restricted nonlinear filters-seem more important than their differences, when modeling human visual representations.

Enhanced Attention Using Head-mounted Virtual Reality.

Abstract: Some evidence suggests that experiencing a given scenario using virtual reality (VR) may engage greater attentional resources than experiencing the same scenario on a 2D computer monitor. However, the underlying neural processes associated with these VR-related effects, especially those pertaining to current consumer-friendly head-mounted displays of virtual reality (HMD-VR), remain unclear. Here, two experiments were conducted to compare task performance and EEG-based neural metrics captured during a perceptual discrimination task presented on two different viewing platforms. Forty participants (20-25 years old) completed this task using both an HMD-VR and traditional computer monitor in a within-group, randomized design. Although Experiment I (n = 20) was solely behavioral in design, Experiment II (n = 20) utilized combined EEG recordings to interrogate the neural correlates underlying potential performance differences across platforms. These experiments revealed that (1) there was no significant difference in the amount of arousal measured between platforms and (2) selective attention abilities in HMD-VR environment were enhanced from both a behavioral and neural perspective. These findings suggest that the allocation of attentional resources in HMD-VR may be superior to approaches more typically used to assess these abilities (e.g., desktop/laptop/tablet computers with 2D screens).

Perturbing Neural Representations of Working Memory with Task-irrelevant Interruption.

Abstract: Working memory maintains information so that it can be used in complex cognitive tasks. A key challenge for this system is to maintain relevant information in the face of task-irrelevant perturbati...

Brain damage associated with impaired sentence processing in acute aphasia

Abstract: Left-hemisphere brain damage commonly affects patients' abilities to produce and comprehend syntactic structures, a condition typically referred to as "agrammatism." The neural correlates of agrammatism remain disputed in the literature, and distributed areas have been implicated as important predictors of performance, for example, Broca's area, anterior temporal areas, and temporo-parietal areas. We examined the association between damage to specific language-related ROIs and impaired syntactic processing in acute aphasia. We hypothesized that damage to the posterior middle temporal gyrus, and not Broca's area, would predict syntactic processing abilities. One hundred four individuals with acute aphasia (<20 days poststroke) were included in the study. Structural MRI scans were obtained, and all participants completed a 45-item sentence-picture matching task. We performed an ROI-based stepwise regression analyses to examine the relation between cortical brain damage and impaired comprehension of canonical and noncanonical sentences. Damage to the posterior middle temporal gyrus was the strongest predictor for overall task performance and performance on noncanonical sentences. Damage to the angular gyrus was the strongest predictor for performance on canonical sentences, and damage to the posterior superior temporal gyrus predicted noncanonical scores when performance on canonical sentences was included as a cofactor. Overall, our models showed that damage to temporo-parietal and posterior temporal areas was associated with impaired syntactic comprehension. Our results indicate that the temporo-parietal area is crucially implicated in complex syntactic processing, whereas the role of Broca's area may be complementary.

Network Localization of Executive Function Deficits in Patients with Focal Thalamic Lesions.

Abstract: The human thalamus has been suggested to be involved in executive function, based on animal studies and correlational evidence from functional neuroimaging in humans. Human lesion studies, examinin...

Does Closing the Eyes Enhance Auditory Attention? Eye Closure Increases Attentional Alpha-Power Modulation but Not Listening Performance.

Abstract: In challenging listening conditions, closing the eyes is a strategy with intuitive appeal to improve auditory attention and perception. On the neural level, closing the eyes increases the power of ...

Neural Dynamics of Serial Dependence in Numerosity Perception.

Abstract: Serial dependence—an attractive perceptual bias whereby a current stimulus is perceived to be similar to previously seen ones—is thought to represent the process that facilitates the stability and ...

Brain Regions Involved in Conceptual Retrieval in Sighted and Blind People

Abstract: If conceptual retrieval is partially based on the simulation of sensorimotor experience, people with a different sensorimotor experience, such as congenitally blind people, should retrieve concepts...

Holding On to the Past: Older Adults Show Lingering Neural Activation of No-Longer-Relevant Items in Working Memory.

Abstract: Goal-relevant information can be maintained in working memory over a brief delay interval to guide an upcoming decision There is also evidence suggesting the existence of a complementary process: