Summary To form a coherent percept of the environment, our brain combines information from different senses. Such multisensory integration occurs in higher association cortices; but supposedly, it also occurs in early sensory areas. Confirming the latter hypothesis, we unequivocally demonstrate supra-additive integration of touch and sound stimulation at the second stage of the auditory cortex. Using high-resolution fMRI of the macaque monkey, we quantified the integration of auditory broad-band noise and tactile stimulation of hand and foot in anaesthetized animals. Integration was found posterior to and along the lateral side of the primary auditory cortex in the caudal auditory belt. Integration was stronger for temporally coincident stimuli and obeyed the principle of inverse effectiveness: greater enhancement for less effective stimuli. These findings demonstrates that multisensory integration occurs early and close to primary sensory areas and—because it occurs in anaesthetized animals—suggests that this integration is mediated by preattentive bottom-up mechanisms.

Integration of touch and sound in auditory cortex

Norville State School aims to achieve the very best educational outcomes for every student through a supportive and challenging environment which focuses on positive attitudes, quality behaviour and learning performance. Norville strives to achieve its aim by stressing the values of care and concern, courtesy, cooperation, commitment, common sense and respect. We aim to develop the confidence of the community that Norville State School serves. Our motto is 'Growing and Learning Together'. We aim to produce students who are intellectually, socially, emotionally and physically capable of functioning effectively in society. We strive to develop positive attitudes towards self, towards others and towards learning.

2015 Annual Report

Real-world environments are typically dynamic, complex, and multisensory in nature and require the support of top-down attention and memory mechanisms for us to be able to drive a car, make a shopping list, or pour a cup of coffee. Fundamental principles of perception and functional brain organization have been established by research utilizing well-controlled but simplified paradigms with basic stimuli. The last 30 years ushered a revolution in computational power, brain mapping, and signal processing techniques. Drawing on those theoretical and methodological advances, over the years, research has departed more and more from traditional, rigorous, and well-understood paradigms to directly investigate cognitive functions and their underlying brain mechanisms in real-world environments. These investigations typically address the role of one or, more recently, multiple attributes of real-world environments. Fundamental assumptions about perception, attention, or brain functional organization have been challenged-by studies adapting the traditional paradigms to emulate, for example, the multisensory nature or varying relevance of stimulation or dynamically changing task demands. Here, we present the state of the field within the emerging heterogeneous domain of real-world neuroscience. To be precise, the aim of this Special Focus is to bring together a variety of the emerging "real-world neuroscientific" approaches. These approaches differ in their principal aims, assumptions, or even definitions of "real-world neuroscience" research. Here, we showcase the commonalities and distinctive features of the different "real-world neuroscience" approaches. To do so, four early-career researchers and the speakers of the Cognitive Neuroscience Society 2017 Meeting symposium under the same title answer questions pertaining to the added value of such approaches in bringing us closer to accurate models of functional brain organization and cognitive functions.

https://par.nsf.gov/servlets/purl/10094971

Are We Ready for Real-world Neuroscience?

There has been longstanding interest from both experimental psychologists and cognitive neuroscientists in the potential modulatory role of various top-down factors on multisensory integration/perception in humans. One such top-down influence, often referred to in the literature as the ‘unity assumption’, is thought to occur in those situations in which an observer considers that various of the unisensory stimuli that they have been presented with belong to one and the same object or event (Welch & Warren, 1980). Here, we review the possible factors that may lead to the emergence of the unity assumption. We then critically evaluate the evidence concerning the consequences of the unity assumption from studies of the spatial and temporal ventriloquism effects, from the McGurk effect, and from the Colavita visual dominance paradigm. The research that has been published to date using these tasks provides support for the claim that the unity assumption influences multisensory perception under at least a subset of experimental conditions. We then consider whether the notion has been superseded in recent years by the introduction of priors in Bayesian causal inference models of human multisensory perception. We suggest that the prior of common cause (that is, the prior concerning whether the multisensory signals originated from the same source or not) offers the most useful way to quantify the unity assumption as a continuous cognitive variable.

/pdf/assessing-the-role-of-the-unity-assumption-on-multisensory-1292axu0e8.pdf

Assessing the Role of the 'Unity Assumption' on Multisensory Integration: A Review.

At any given moment, we receive input through our different sensory systems, and this information needs to be processed and integrated. Multisensory processing requires the coordinated activity of distinct cortical areas. Key mechanisms implicated in these processes include local neural oscillations and functional connectivity between distant cortical areas. Evidence is now emerging that neural oscillations in distinct frequency bands reflect different mechanisms of multisensory processing. Moreover, studies suggest that aberrant neural oscillations contribute to multisensory processing deficits in clinical populations, such as schizophrenia. In this article, we review recent literature on the neural mechanisms underlying multisensory processing, focusing on neural oscillations. We derive a framework that summarizes findings on (1) stimulus-driven multisensory processing, (2) the influence of top-down information on multisensory processing, and (3) the role of predictions for the formation of multisensory perception. We propose that different frequency band oscillations subserve complementary mechanisms of multisensory processing. These processes can act in parallel and are essential for multisensory processing.

Neural Oscillations Orchestrate Multisensory Processing.

Encoding the temporal properties of external signals that comprise multimodal events is a major factor guiding everyday experience. However, during the natural aging process, impairments to sensory processing can profoundly affect multimodal temporal perception. Various mechanisms can contribute to temporal perception, and thus it is imperative to understand how each can be affected by age. In the current study, using three different temporal order judgement tasks (unisensory, multisensory, and sensorimotor), we investigated the effects of age on two separate temporal processes: synchronization and integration of multiple signals. These two processes rely on different aspects of temporal information, either the temporal alignment of processed signals or the integration/segregation of signals arising from different modalities, respectively. Results showed that the ability to integrate/segregate multiple signals decreased with age regardless of the task, and that the magnitude of such impairment correlated across tasks, suggesting a widespread mechanism affected by age. In contrast, perceptual synchrony remained stable with age, revealing a distinct intact mechanism. Overall, results from this study suggest that aging has differential effects on temporal processing, and general impairments with aging may impact global temporal sensitivity while context-dependent processes remain unaffected.

/pdf/aging-impairs-temporal-sensitivity-but-not-perceptual-340v5amx4b.pdf

Aging Impairs Temporal Sensitivity, but not Perceptual Synchrony, Across Modalities.

The causal relationship between a voluntary movement and a sensory event is crucial for experiencing agency. Sensory events must occur within a certain delay from a voluntary movement to be perceived as self-generated. Therefore, temporal sensitivity, i.e., the ability to discriminate temporal asynchronies between motor and sensory events, is important for sensorimotor binding. Moreover, differences in the physical propagation of external stimuli can sometimes challenge sensorimotor binding, generating illusory asynchrony. To overcome this problem, the brain adjusts the perceptual timing of sensory and motor events. This mechanism, named sensorimotor recalibration, helps keeping causality judgments accurate. As humans age, the broad decline in sensory and motor processing may reduce temporal sensitivity, and compromise sensorimotor recalibration. In the current study, we investigated the effect of aging on sensorimotor temporal binding by measuring changes in both temporal sensitivity and recalibration. Young and elderly adults were exposed to a prolonged physical delay between a voluntary movement (a keypress) and its perceptual consequence (a visual stimulus). Before and after this exposure, participants performed a sensorimotor temporal order judgment (TOJ) task. As expected, elderly adults showed reduced sensorimotor recalibration and sensitivity as compared to young adults, suggesting that aging affects sensorimotor temporal binding.

/pdf/age-related-changes-in-sensorimotor-temporal-binding-1x2nmhp1pc.pdf

Age-Related Changes in Sensorimotor Temporal Binding.

Spatial modulation of motor-sensory recalibration in early deaf individuals.

Prolonged adaptation to delayed sensory feedback to a simple motor act (such as pressing a key) seems to provoke a recalibration of sensory-motor synchronization, causing instantaneous feedback to appear to precede the motor act that caused it (Stetson et al., Neuron, 2006). We investigated whether a similar recalibration occurs in school-age children, when plasticity may be expected to be even greater than in adults. Subjects adapted to delayed feedback for 100 trials, intermittently pressing a key that caused a tone to sound after a 200 ms delay. During the test phase, subjects responded to a light cue by pressing a key, which was accompanied by a tone played at a variable interval before or after the keypress. Subjects judged whether the tone preceded or followed the keypress. The data were fit with cumulative gaussian psychometric functions, yielding estimates of the points of perceived synchrony (mean) and precision (width). In agreement with previous studies, adults showed a shift in perceived synchrony after adaptation, so the keypress appeared to trail the auditory feedback, implying sensory-motor recalibration. However, school children of all ages (7, 8, 10 & 12 years) showed no adaptation of perceived simultaneity, although their precision in the simultaneity task was comparable with that of adults. This suggests that school-age children do not recalibrate sensory-motor simultaneity after adaptation, implying that the neural mechanisms underlying recalibration require the maturity of both the motor and sensory systems.

Children do not recalibrate motor-sensory temporal order after exposure to delayed sensory feedback

We recently showed that congenitally blind adults have severely impaired thresholds in an auditory spatial bisection task, pointing to the importance of vision in constructing auditory spatial maps. To explore strategies that may improve the auditory spatial sense of the blind, we investigated the impact of tactile feedback on spatial auditory localization in 29 blindfolded sighted subjects, assigned at random to one of three groups: tactile feedback ( n = 11 ), verbal feedback ( n = 8 ) and no feedback ( n = 10 ). For all groups, auditory bisection thresholds were first measured by playing three consecutive sounds at 500 ms intervals, and subjects judged whether the second sound was spatially closer to the first or to the third. The tactile-feedback group was given two audio-tactile training sessions of 100 trials, where each auditory trial was followed by the same spatial sequence played on the subject’s forearm; auditory spatial bisection thresholds were evaluated after each session. The no-feedback group did the same sequence of trials, with no feedback. In the verbal-feedback condition, the positions of the sounds was verbally reported to the subject after each training trial. Interestingly, subject performance improved significantly only after the audio-tactile training, on average by a factor of 2.5. The no-feedback sessions, and those with verbal feedback produced no statistically significant improvement. The results suggest that direct tactile feedback interacts with the auditory spatial localization system, possibly by a process of cross-sensory recalibration. More generally, the results encourage the possibility of designing rehabilitation programs to help blind individuals establish a robust sense of space.

Tiziana Vercillo

Papers

Aging Impairs Temporal Sensitivity, but not Perceptual Synchrony, Across Modalities.

Age-Related Changes in Sensorimotor Temporal Binding.

Spatial modulation of motor-sensory recalibration in early deaf individuals.

Children do not recalibrate motor-sensory temporal order after exposure to delayed sensory feedback

Tactile feedback improves auditory spatial localization